1

I want to detect the scrolling direction of another application. I am able to do so if the user scrolls through Mouse Wheel or Keyboard (Up/Down/Left/Right keys) through hooks. But I am not able to capture the same when the user uses the scrollbars present on the applications like Chrome.

I've tried below native method, but it does not work for many applications like chrome and works for Notepad++, as it is not able to detect scroll bars for chrome. I want something that works for all the applications.

[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool GetScrollInfo(IntPtr hwnd, int fnBar, ref SCROLLINFO lpsi);

I have quite a bit of research but could not find anything that could give me the directions in which the page is scrolling. Please let me know if any further information is required.

Update:

I am trying to use UI Automation to get the scroll bar information for Chrome.

Here's how?

I have made a collection of windows Gui Rectangles using EnumChildWindows, which retrieved child controls as well. Based upon the mouse position, have selected the window handle whose Gui Rectangle contains my mouse position. The handle I obtained had the Gui Rectangle = chrome's client area.

Problem:

Below is the code. And it gives me an empty collection in elementCollection in case of Chrome and successfully returns 2 scroll bar elements in case of Notepad++.

var element = AutomationElement.FromHandle(handle);
if(element != null)
{
    Condition condition = 
       new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.ScrollBar);

    // Find all children that match the specified conditions.
    AutomationElementCollection elementCollection =
        element.FindAll(TreeScope.SubTree, condition);
 }
shruti singh
  • 146
  • 11
  • What would you use this information to *do*? Are you sure you're thinking about your problem at the right *level*? E.g. if this is some sort of accessibility tool, there are higher level ways of interacting with other applications that let you work at a more "semantic" level than trying to analyse scroll movements yourself. – Damien_The_Unbeliever Nov 14 '18 at 09:45
  • I want to do a panoramic capture for applications. User can go to the application and scroll, and I would be taking screen shots, and later would be combining all the screenshots to make a single image. The requirement is such that the user has to manually scroll, and I can't use `SendMessage(IntPtr hWnd, int Msg, int wParam, int lParam);` to simulate scroll bar movements. – shruti singh Nov 14 '18 at 10:54

1 Answers1

0

What you need to do is call the GetScrollInfo method twice and compare those values to find out; whether the control is being scrolled and to calculate (previous < current) the direction. You need to repeat this process as long as you want to keep monitoring the scrolling direction.

Now you should create some kind of worker class to handle things nicely, for the purpose of this question I've used a simple Console application that polls the SCROLLINFO structure of the handle using a while-loop:

static void Main(string[] args)
{
    var handle = (IntPtr)0x35087C; // notepad textbox handle

    var prevInfo = default(SCROLLINFO);
    int iterations = 1000;

    while (iterations > 0)
    {
        var curInfo = new SCROLLINFO();
        curInfo.cbSize = Marshal.SizeOf(curInfo);
        curInfo.fMask = (int)ScrollInfoMask.SIF_ALL;

        if (GetScrollInfo(handle, (int)ScrollBarDirection.SB_VERT, ref curInfo))
        {
            var dir = curInfo.nPos < prevInfo.nPos ? "Up" : "Down";
            dir = curInfo.nPos == prevInfo.nPos ? "Unchanged" : dir;

            Console.WriteLine(dir);

            prevInfo = curInfo;
        }
        else
        {
            Console.WriteLine($"No scrolling.. {Marshal.GetLastWin32Error()}");
        }

        System.Threading.Thread.Sleep(100);
        iterations--;
    }

    Console.ReadLine();
}

Note that the handle variable contains the pointer to the textbox of a notepad process (which is NOT the same as the handle to the notepad process). I've used WinSpy++ to retrieve the handle from the notepad process.

I've used the GetScrollInfo MSDN page to read about the API call itself, GetScrollInfo Pinvoke.NET page to obtain all the required structs/enums.

Ashish Kamble
  • 2,555
  • 3
  • 21
  • 29
Jevgeni Geurtsen
  • 3,133
  • 4
  • 17
  • 35
  • Can you please try for chrome? Your solution gave me 1447 error for chrome. Some application's do not respond to GetScrollInfo and give error 1447 which is ERROR_NO_SCROLLBARS aka "The window does not have scroll bars". https://www.experts-exchange.com/questions/23496918/Problem-with-GetScrollInfo.html – shruti singh Nov 14 '18 at 10:47
  • @shrutisingh Jevgeni has already done the heavy lifting of showing you how to do the task. You need to get your specifics yourself. Even if he did it with chrome, you will not get the same result because each process has it's own handle and you having the exact same handle as Jevgeni would be a major coincidence. As for why the method isn't working in chrome's instance the obvious answer would be you've got the wrong instance. My guess would be you're checking the main chrome instance instead of renderer which it's likely happening. – Maverik Nov 14 '18 at 11:03
  • @Maverik My issue is this only that this logic does not work for all the child handles. I am aware that this works for Notepad as I had already tried it. In case of chrome, I replaced the handle with the child handle of chrome, and it is not able to get the scroll info. And I am getting the child handle through `EnumChildWindows` which is correct as I am able to simulate `SendMessage(IntPtr hWnd, int Msg, int wParam, int lParam)` using the same handle. – shruti singh Nov 14 '18 at 11:08
  • You should've added that you where targeting chrome, this is crucial information that is missing from your main post. Chrome uses D3D (DirectX 3D) rendering which is a whole lot different in comparison with a simple control, like notepads' textbox. Knowing this I doubt you can use the `GetScrollInfo` API. You need to hook chrome and find out where they store the scroll information in-memory and work from there. – Jevgeni Geurtsen Nov 14 '18 at 11:12
  • 2
    @mav: That all sounds convincing, except, it's wrong. Chrome doesn't use native windows for anything (other than a frame window and a rendering host). Everything you see, however, is done using custom widget implementations. The scrollbars may look like native scrollbars, but they aren't either one of the [two types of scrollbars](https://blogs.msdn.microsoft.com/oldnewthing/20040510-00/?p=39413). As such, they will not respond to messages or the regular windowing API. You have to go for a different solution here, e.g. UI Automation. – IInspectable Nov 14 '18 at 11:16
  • @JevgeniGeurtsen I need a solution that works for all the applications which can have scrollbars. Be it notepad or chrome or IE. Snagit does the same for Panoramic captures. I am not sure how it does it, but I am trying to achieve the same. – shruti singh Nov 14 '18 at 11:39
  • @IInspectable Can you please suggest a few links for UI Automation given my requirement? – shruti singh Nov 14 '18 at 12:50
  • @shr: [Accessibility](https://learn.microsoft.com/en-us/dotnet/framework/ui-automation/index). – IInspectable Nov 14 '18 at 13:51
  • @IInspectable : With UI Automation as well, I am able to get the Horizontal and Vertical scrollbars of Notepad++ only, and not for Chrome. Can you please try out my code for these 2 separate applications with appropriate handle and let me know where I am going wrong? – shruti singh Nov 15 '18 at 06:52
  • @shrutisingh Are you sure the application you have mentioned uses this technique? If its solely about **recording** the screen, then they would probably use a WinAPI, like `SendKeys`, so scroll through a page.. I doubt they will actually read the scrollbars -- they'll probably just check the difference between frames, and if nothing has changed after the `SendKeys` call they know they are at the end (or start) of a page. – Jevgeni Geurtsen Nov 15 '18 at 13:38
  • @JevgeniGeurtsen : I am not sure what exactly they do. It's not exactly recording the screen, it's actually taking multiple screenshots while the page is scrolling, and combines all of them into a single image. eg: https://hub.mangoapps.com/sf/MTUzMjMwXzE2NDYxMTA – shruti singh Nov 17 '18 at 09:12
  • I understand, so why would you need to read the scrollbar information to accomplish that? As said before; one could just take a few screenshots while scrolling (`SendKeys`) and use some sort of delta processing to find out where the page ends (== no changes on sequential screenshots = page end). I think you are overdeveloping/thinking the problem. – Jevgeni Geurtsen Nov 20 '18 at 13:02