How to Find Exact match of an Image in Sikuli with Java

Question

Am new to Sikuli and trying to Automate Citirx Application. Need Help

Am trying to select a user role in a screen, The screen has multiple roles and hence i need to scroll down the screen and search for a particular Role and click the Role.

I have Captured image of a Particular Role that i need to select and used below Code. In the second Image i have highlighted the Role i need to select in Red

Below is the Code an Trying:

Creating a Method:

 public static boolean clipExist(Screen screen, String clip )
 {
        Match m = screen.exists(clip);
         if(m != null)
      {
              return true;
        }
       else
       {
          return false;
       }
   }

Using the Method:

        while(! clipExist(screen, "C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\DownArrow.PNG"))       
    {           
       screen.wheel(1 , 3);     
       if(clipExist(screen, "C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\Roles\\UK\\ENTP\\GEDIS_SALES_SUPPORT_ORL_CPF2.0_UK_ENTP.PNG"))
       {
        screen.doubleClick("C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\Roles\\UK\\ENTP\\GEDIS_SALES_SUPPORT_ORL_CPF2.0_UK_ENTP.PNG",0);
        break;
       }
      }

Simple: i need to scroll down the screen to find a particular image and click on the image. — Satish Dhanaraj, Nov 20 '15 at 15:07
Yes but what stops you from doing that? Your code doesn't work or you don't know how ho do something? — Eugene S, Nov 23 '15 at 02:41
The Above code which i used is not working as expected. First it checks for the image and since the image is not available it scrolls the page, still the expected image is not present, but my code goes and clicks some other image and proceeds which is wrong. — Satish Dhanaraj, Nov 24 '15 at 05:56
You will have to provide more information. If your script behavior is different from what you expect, explain how. If you are getting errors/exceptions, add them to your question. Unless you have explained in details, it will be impossible to answer your question. — Eugene S, Nov 24 '15 at 05:59

score 2 · Answer 1 · answered Feb 22 '16 at 13:39

The image recognision uses per default a similarity of 0.7 (see description of Patterns in SikuliX Documentation). That means SikuliX looks for 'pretty similar' images. You can specify the similarity for the pattern recognision thanks to the method similar, or in your case use the method exact. In your method clipExist, you should replace the name of the image:

    Match m = screen.exists(clip);

by:

    Match m = screen.exists(Pattern(clip).exact())

score 0 · Answer 2 · answered Dec 07 '15 at 13:25

0

It seems SikuliX 1.1 experience some problem with finding the text on a screen, but recognition works. You might want to scan the entire text screen by screen and split the lines. Next compare each line with the required role and save the degree of similarity. Select the line with the biggest similarity. In Python/Jython exists a special function for that in difflib module. similarity = difflib.SequenceMatcher(None, string_a, string_b)

answered Dec 07 '15 at 13:25

IgorG

1
2

Text recognition in Sikuli is generally unreliable at the moment. This is true for all versions. – Eugene S Dec 14 '15 at 03:02

score 0 · Answer 3 · answered Mar 17 '16 at 23:21

Here are the alternatives that you can do.

First alternative: capture scrollbar

Capture the down arrow in the scrollbar
Capture the image when you reach the end of scrollbar. The image contains the scroll progress and the down arrow of the scrollbar
Click down arrow until you find image of (2)

This method have drawback i.e. when the number of items are dynamic, the visual appearance of (2) will be different especially the scroll progress. However, this can be tricked by capturing only the lower part of scroll progress and the arrow. Please note that your mouse may make difficulty in (3) because you may not find (2) when it is covered by mouse. To handle this, every time you click down arrow, you may hover your mouse a bit before checking for (2). This is the complete script:

down_arrow = "downarrow.png"
complete_scroll = "completescroll.png"

while not exists(complete_scroll):
    click(down_arrow)
    hover(Location(300, 200))

Second alternative, use keyboard (down key)

Click anywhere in the items to be scrolled and do some type(Key.DOWN) for the number of item you have. In case you have dynamic number of item, you may do type(Key.DOWN) for any number that always bigger than your number of items. Here is the script to do

inside_item = "inside.png"

for n in range(10000):
    type(Key.DOWN)

Hope it helps

score 0 · Answer 4 · edited Aug 09 '19 at 14:02

0

I used 's' as a screen class reference. Hence, once we get an image then we will set the region for the same followed by the required image where you want to click

public static void main(String args[])
{
    Match m = s.find("IMAGE");
    Region r = new Region(m.x+11, m.y+22,12,12);
    r.click(); 
    s.find("ENTPIMAGE.PNG");
    r.click("ENTPIMAGE.PNG");
}

edited Aug 09 '19 at 14:02

Suraj Rao

29,388
11
94
103

answered Aug 09 '19 at 13:51

inderpreet

1
1

How to Find Exact match of an Image in Sikuli with Java

4 Answers4