How can Sikuli be used to wait for a button for a long time, with perhaps some maintenance task in between?

Question

I have a webpage where I am waiting for a button to appear, and when it appears I would like to click it. The button is on a timer and may take as long as an hour to appear. Also, if the button takes longer than a certain length of time to appear, I'd like to move the mouse (otherwise the website will log me out automatically).

So, to wait for a button to appear I devised this Sikuli script:

button = "button.png"

while(1):
    if exists(button):
        print("found it")
        click(button)
        break
    else:
        print("wait longer")
        wait(button,30*60)
        # do a regular task

print "all done!"

The above does not seem to be functional. If the button is on screen, the script will find it... However, if it has to wait it will simply time out quickly with a FindFailed exception (on the click() even though the button does not exist on screen). I considered writing a handler, but seems like overkill.

What am I doing wrong and what is the best way to wait a long period for a visual event like this?

Why are you automating web sites with sikuli? There are better tools for that, like Selenium or Watir. — Željko Filipin, Jul 26 '12 at 07:08
@ŽeljkoFilipin tools like Selenium (functional testing) are good. But it not only adds value but gives more quality, increases productivity when GUI based testing like sikuli is complemented with/without functional testing. — Sree Rama, Nov 07 '13 at 06:09

score 10 · Accepted Answer · edited Dec 27 '12 at 10:20

10

Some other thoughts for you...

 while(1):
 wait(Button, 30*60) # This will spinlock for 30 minutes for the button to appear
 if exists(Button):
     hover(Button) # Debug statement allowing user to see what Sikuli has matched to
     click (Button)
 else:
     mouseMove(Location(50,100))
     mouseMove(Location(50,200))

Links:

edited Dec 27 '12 at 10:20

RaiMan

1,088
8
13

answered Jul 26 '12 at 18:33

spearson

1,016
8
18

Is it really good approach using "while(1)"? It works for first time for me. I understand that it is better to apply : Observing Visual Events in a Region as suggested by @Alessandro Da Rugna. http://doc.sikuli.org/region.html?highlight=observ#Region.observe – Sree Rama Nov 07 '13 at 06:28

score 6 · Answer 2 · edited Dec 27 '12 at 10:48

Maybe Sikuli recognizes something that looks quite your button, and tries to click it. If you right click in the IDE your button pattern, you can fine tune the tolerance level for recognition. Try to cut the image exactly around your button and increase the value to be more precise.

I suggest you to read this tutorial
http://doc.sikuli.org/tutorials/surveillance/surveillance.html
and to set up a event handler to manage your button when it appears
http://doc.sikuli.org/region.html#Region.onAppear
http://doc.sikuli.org/region.html#observingvisualeventsinaregion
It is not much code to write.

You can get a nice example with full source code in Sikuli's Blog here http://sikuli.org/blog/2011/08/15/sikuli-plays-angry-birds-on-google-games/

I think you can just set up your handlers and go with

observe(FOREVER)

The angry birds demo is a really great analog for this problem. — spearson, Jul 26 '12 at 18:36

score 2 · Answer 3 · answered Mar 04 '15 at 19:19

If you want sikuli to do stuff while your waiting for an image i would use the onAppear(pic, function) and observe(FOREVER, true) methods this is how it works

event = Sikuli.event

def function(event):
    click(yourButton.png)

onAppear(picYourWaitingFor.png, function)
observe(FOREVER, true)

basically what this does is onAppear will continuously scan the screen for picYourWaitingFor.png. sikuli continues execution after words so it's scanning while its working. on the appearance of said pic it will jump to the function you put down as the second parameter of onAppear.

score 1 · Answer 4 · edited Jan 01 '18 at 19:05

I have this same issue as described. Its not about waiting forever. And Observe won't work either, because that does watch forever. Think about wanting to check for event only for a certain period of time say 60 seconds. If it doesn't occur, move on. This could be happening in a specific series of events. If the image doesn't appear in the 60 seconds, move on to do another series.

wait(image,60)

...will crash after 60 seconds if it doesn't find the image, which isn't what is wanted at all in my case.

So I did something like this:

attempt = 1
count=0
while attempt:
     if exists(image):
         attempt=0
     else:
        count=count+1
        if count>60:
          attempt=0
        else:
           wait(1)

Probably a better way and doesn't give an exact time, but approach doesn't crash the script.

You could also try: except it.. Should be shorter.

How can Sikuli be used to wait for a button for a long time, with perhaps some maintenance task in between?

4 Answers4