9

I have to admit that I fell in love with Selenium for its record-and-play feature as well as the testcase generation functionality for those recorded actions from the IDE. But I am still hesitated to advance to the implementation stage because of the incidental details (e.g, locating the events with DOM, xpath..etc) that are built into the testcase during the recording, which could make the testcase failure prone whenever there is a html change once it's imported to the RC. I fully understand that it's a part of testers' jobs to adjust the expected results from time to time as part of the regression test, but I also do not wish the time spent on this is larger than the time that takes to do the manual test.

As far as I know Selenium with Robot framework has the keywords form of testcases. My guess is it allows us to extract the incidental details into various keywords, which could make the testcases being adjusted easier and are more maintainable. (Please correct me if I am wrong)

It will be appreciated to hear suggestions on how an effective UI automation environment should be setup. Should I just use Selenium RC or Selenium with Robot framework? And why?

Thanks in advance

Bryan Oakley
  • 370,779
  • 53
  • 539
  • 685
Daniel
  • 631
  • 10
  • 18
  • Or,anyone ever use sikuli? how is it comparing to selenium ide? can the testcases be easily integrated to test runners? – Daniel Feb 18 '11 at 19:23

2 Answers2

10

You are absolutely right that incidental and often changing details in the produced scripts is the biggest problem of record-and-playback automation. You can obviously remove the details from the scripts after recording, but in my opinion it's better to build reusable libraries and code scripts manually from the start.

A good alternative for coding scripts using "real" programming languages is using some higher level automation framework such as Robot Framework that you mentioned. As you speculated, Robot's reusable keywords and also variables make extracting details away from tests very easy. The test cases in SeleniumLibrary's demo illustrates this very well and the demo also shows how to use Selenium through Robot.

You also asked about Sikuli. I've never used it myself but it sure looks interesting. You might be interested on this great how-to that explains how to use it through Robot Framework.

Pekka Klärck
  • 2,348
  • 2
  • 18
  • 19
  • So, you say it is better to write tests in Robot Framework straight away? I have experience in writing tests in Robot (and I like it), but I haven't tried the Selenium IDE at all. It does seem easier or faster to use.. – Qwerty Sep 11 '14 at 11:48
  • 1
    If your target is to create Robot Framework tests, then it's definitely better to create them using Robot syntax from the beginning. You cannot, for example, easily convert Selenium IDE tests to Robot syntax. Tests generated by Selenium IDE are very low level and hard to maintain in general, but obviously there are use cases where that's fine. – Pekka Klärck Sep 23 '14 at 21:14
2

Our company is using Fitnesse, not Robot, to control Selenium however, we have the same problem. We switched from making assumptions about the DOM to only accessing elements by ID. Since this is cumbersome in Fitnesse we are currently working to add a Selenium backend to our own Framework (which previously only had backends for Java and Smalltalk).

So, by requiring that elements with certain ID's are present in the DOM we will of course break our tests if someone removes the elements from the page; however, we found that this behavious is very useful as this enforces the contract the tests made with the implementation and it is a good thing we find missing elements as soon as someone broke the implementation.

In addition, it is good practice to keep UI automation skin-deep: Only test what is present on the page with Selenium and test the business-logic by calling the underlying functions directly.

Bernd Elkemann
  • 23,242
  • 4
  • 37
  • 66
  • Correct me if I am wrong, sounds like your company enforced web developers to mine ID's into element tags. And, if the ID's are the contract between testers and developers, this will greatly reduce the risk of introducing incidental detials on the testcases. Moreover, time spent on fixing it will really because of the requirement change. May I ask how QA engineers in your our company generate their testcases, with Selenium IDE or simply writing the code out. And what is the benefit to this option versus the other. Looking forward to your reply. Thanks – Daniel Feb 25 '11 at 03:33
  • 2
    Yes, we use ID's per DOM-Element. BUT these can be auto-generated and both developers and testers can easily find them because they are pasted into the raw html (no indirection, and this is only for our testing not for external consumption). Once a tester has an ID he needs no longer navigate the DOM-tree but can just select-by-ID and write a test-case in table-form like |123|contains|asdf|, and if one of the test-tables notices an error, the testsuite can give the element ID to the developers who immediately know the exact place they broke. – Bernd Elkemann Feb 25 '11 at 11:22
  • it's being a long time since I hit back this page. Instead of creating a bug aand hoping developers to inject id at correct place, myself as a tester do it myself. I do it myself and judge myself if I can touch the production code by checking if my integration test covers the changed code or not. Developer has their time occupied. Figured this would get the work done faster and more pleasant work env. – Daniel Jan 23 '13 at 04:16