Unable to Pass Selenium tests within Azure Pipeline

Question

I am trying to build out UI test cases in our CI/CD build pipeline within Azure Devops and I can't seem to workaround a basic use case where we route to one of our internal web pages landing screen, and verify a Div with a Selector exists.

        [Theory]
        [InlineData("Chrome")]
        public void TestFindElementByID(string browser)
        {
            var driver = SetupDriver(browser);
            try
            {
                driver.Navigate().GoToUrl(TestConfig.WebURL);
            
                WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
                IWebElement element = wait.Until(ExpectedConditions.ElementIsVisible(By.Id("mainRepDiv")));
                //Thread.Sleep(2500);
                //IWebElement element = driver.FindElement(By.Id("mainRepDiv"));
                Assert.True(element != null);
            }
            catch (Exception ex)
            {
                
            }
            finally
            {
                driver.Quit();
            }
            

        }

In my web page, I have a basic div that I just want to see that it exists once the chromedriver navigates to the URL.

Element To Check

And here is a log of the Azure Pipeline failing on this test. Pipeline Log

I do have an additional test that just asserts that the Browser is able to go to the URL, and the URL is in fact what I routed to, and that test passes.

driver.Navigate().GoToUrl(TestConfig.URL);
Assert.True(driver != null);
Assert.True(driver.Url.ToLower() == TestConfig.URL.ToLower());

There also some layers that likely are making this more difficult such as The Azure Pipeline is running off a Self-hosted agent.

This self hosted agent is also the same server that is running the application were trying to test against.
There is OS level Security that shows up if you route to it normally, however as the Windows Server that is running the application would have a valid credential to the browser, it should bypass this dialog.

So things I have tried so far:

Various ways of actually selecting the Element incase that was off, such as By ID that is shown, or by XPath as well.
Using the Thread.Sleep() or the WebDriver Wait commands to determine if it was a page load.
Implementing AutoIT Process that will run in the event there is in fact an OS popup that displays, so I can prefill the Admin Username/Password credentials incase it actually is not getting there.
Running the unit test locally on my client PC, publishing the code to the Server, running the code from the application server directly and verifying it works there as well.

So I'm a bit at a loss what to check on regarding the Azure Pipeline build itself and why the ChromeDriver is unable to pass this test.

Finally - if this is also helpful, here is a screenshot of the YML file pertaining to the Tasks.

- task: VSTest@2
  displayName: "Run UI Tests"
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\Project.Test.UI.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'
    uiTests: true

Please do not post images of code. Instead, [edit] you question to include the code in code blocks. It is much easier to read, and people can copy the code in order to write an answer. — Greg Burghardt, Mar 26 '22 at 14:19
@LeoLiu-MSFT I have just received a PAT from the DevOps Server Admin and have setup the Agent to run as Interactive, I am going to try to tie the Selenium Task to run as that agent now under our current pipeline tasks and see where I get. Thanks for the help so far. — Robin, Apr 01 '22 at 13:40

score 0 · Accepted Answer · answered Mar 28 '22 at 02:05

Unable to Pass Selenium tests within Azure Pipeline

Please make sure your private agent run as an interactive process.

According to the document Interactive vs. service:

You can run your self-hosted agent as either a service or an interactive process. After you've configured the agent, we recommend you first try it in interactive mode to make sure it works. Then, for production use, we recommend you run the agent in one of the following modes so that it reliably remains in a running state.

As an interactive process with auto-logon enabled. In some cases, you might need to run the agent interactively for production use - such as to run UI tests. When the agent is configured to run in this mode, the screen saver is also disabled. Some domain policies may prevent you from enabling auto-logon or disabling the screen saver. In such cases, you may need to seek an exemption from the domain policy, or run the agent on a workgroup computer where the domain policies do not apply.

So, if your agent not run as interactive mode, please configure a another agent in interactive mode to test this issue.

I was able to create a new test agent that is running interactively. The Pipeline is now executing two jobs. The 2nd job isn't picking up any actual tests yet - job: 'RunUITests' pool: name: 'Server-Test' demands: - agent.name -equals TestAgentInteractive steps: - task: VSTest@2 displayName: 'Run UI Tests' inputs: testSelector: 'testAssemblies' testAssemblyVer2: | **\Test.UI.dll !**\*TestAdapter.dll !**\obj\** searchFolder: '$(System.DefaultWorkingDirectory)' uiTests: true — Robin, Apr 07 '22 at 14:59
Was able to correct the last issue as the new pipeline job needed to build the solution in the new agent/work folder and from there I had the .dll I was looking for to run tests against. Thanks for the help! — Robin, Apr 07 '22 at 16:02

Unable to Pass Selenium tests within Azure Pipeline

1 Answers1