AWT Robot not able to drag a window

Question

I'm trying to move a Windows Explorer window using the AWT Robot. The robot is running in Java 7, and the OS is Windows 7.

I'm able to move the mouse and click on things, but when I try to click-and-drag, it doesn't seem to be pressing the button at all. I can't see what's wrong, or think of how to figure out what's happening.

I started out using Sikuli:

mouse.mouseDown(InputEvent.BUTTON1_MASK);
mouse.drop(targetLocation);

When that didn't work, I tried a lower-level implementation, working with the robot directly:

Robot robot = new Robot();
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseMove(targetLocation.getX(), targetLocation.getY());
robot.mouseRelease(InputEvent.BUTTON1_MASK);

The mouse starts in the correct place and moves to the correct destination, but doesn't seem to press the button.

score 4 · Answer 1 · answered Jun 11 '13 at 18:27

In sikuli use mouse.drag() then mouse.drop(). Example:

ScreenRegion fullScreenRegion=new ScreenRegion();
ImageTarget dragImageTarget=new ImageTarget("dragTargetFile");
ScreenRegion dragTargetRegion=fullScreenRegion.find(dragImageTarget);
ImageTarget dropImageTarget=new ImageTarget("dropTargetFile");
ScreenRegion dropTargetRegion=fullScreenRegion.find(dropImageTarget);

Mouse mouse = new DesktopMouse();
mouse.drag(dragTargetRegion.getCenter());
mouse.drop(dropTargetRegion.getCenter());

For the Java Robot API: You should call mouseMove(), mousePress(), mouseMove(), and then mouseRelease() in that order. Example:

Robot robot=new Robot();
// drag
robot.mouseMove(x1, y1);
robot.mousePress(InputEvent.BUTTON1_MASK);
// drop
robot.mouseMove(x2, y2);
robot.mouseRelease(InputEvent.BUTTON1_MASK);

score 2 · Answer 2 · answered Jul 12 '17 at 16:28

I had the exactly the same problem. Even tried what @Andrzej Kasp said, add thread sleep for certain mount of time. It does not work, until I made some modification based on @Andrzej Kasp `s comments. Thanks to Andrzej Kasp.

The whole reason why it does not work is because the thread got executed in series. I tried to add thread.sleep(100); between each action does not work. Codes as below:

robot.mouseMove(x1,y1);                      //step 1
thread.sleep(100);
robot.mousePress(InputEvent.BUTTON1_MASK);   //step 2
thread.sleep(100);
robot.mouseMove(x2, y2);                     //step 3
thread.sleep(100);
robot.mouseRelease(InputEvent.BUTTON1_MASK); //step 4

It does not work for me. It might work for you though. Depends on the computer. The whole theory behind this is that, if you want to make sure if work, you have to make sure the mouse is pressed while the cursor is moving. ie. step 2 and step 3 are being executed at the same time. In normal case, computer will not create a new thread while running this program unless you make use of thread.sleep(), it will suspend the current thread and processor time will be assigned to other threads. So you do not have to add thread.sleep(100) after each step, instead, you only need to add thread.sleep(100) between step 2 and step 3. I have tested these codes and they worked. Codes as below:

robot.mouseMove(x1,y1);                      //step 1
robot.mousePress(InputEvent.BUTTON1_MASK);   //step 2
thread.sleep(100);
robot.mouseMove(x2, y2);                     //step 3
robot.mouseRelease(InputEvent.BUTTON1_MASK); //step 4

Let me know if you have more questions.

I found that it is dependent on small adjustments; for example, I started out with with your last four steps, but in order to get it to be recognised as a dnd gesture, I had to add one additional tiny intermediate movement. Nevertheless, so far I haven't managed that after the `mouseRelease` the DnD was actually completed. (in my case, the DnD was an object across the GUI, not a window, though) — 0__, Jan 03 '20 at 22:19

score 1 · Answer 3 · answered Jul 04 '13 at 08:38

1

I had a similar problem. Try adding Thread.sleep(100); (or more then 100) between each Robot operation.

answered Jul 04 '13 at 08:38

Andrzej Kasp

177
1
2
8

Marcin Kowalczyk · Answer 4 · 2013-05-16T08:44:57.323

0

Why it didn't work for you with Sikuli? What was a problem? I tried it by myself and it worked. My environment: Win7 with disabled window transparency, Sikuli r930. To simplify a test, some window is already opened before starting a test. This is my working example:

winExpl_topRightCornerImg = Pattern("1368690528515.png").targetOffset(-2,-13) 
"""winExpl_topRightCornerImg is a part of a window containing top bar.
I captured huge rectangle containing navigation arrows and I changed target offset,
so that Sikuli clicks only on a bar"""
anyRectangleAtDesktop = "1368690615156.png" #any particular part of a desktop
drag(winExpl_topRightCornerImg)
dropAt(anyRectangleAtDesktop)

edited May 16 '13 at 08:44

answered May 16 '13 at 08:04

Marcin Kowalczyk

649
6
17

I'm using Sikuli-API, the Java library, rather than Sikuli the Jython wrapper/library. The problem, as I said in my question is "The mouse starts in the correct place and moves to the correct destination, but doesn't seem to press the button." – Nathaniel Waisbrot May 16 '13 at 16:06
@ Nathaniel Waisbrot: I see. We work in different Sikuli environments, that's why behavior is also different. – Marcin Kowalczyk May 16 '13 at 18:55

AWT Robot not able to drag a window

4 Answers4

Linked