1

I am debugging some code that uses a 3rd party 64-bit DLL to access a custom USB device. My environment is Microsoft Visual Studio 2012 on Windows 8.1 x64.

According to an incomplete and unreliable document, the DLL is supposed to issue a USBDEVFS_CONTROL ioctl to read 1 byte from a connected USB device. The definition involves 

ctrl.bRequestType = bmRequestType;
ctrl.bRequest     = bRequest;
ctrl.wValue       = wValue;
ctrl.wIndex       = wIndex;
ctrl.data         = ByteArray;
ctrl.wLength      = 64;
ctrl.timeout      = 1000;

Here bmRequestType, bRequest, wValue, and wIndex are constants provided by the device manufacturer, and ByteArray is a uint8_t[64] buffer that contains the specific command.

The DLL accepts application-specific parameters, packs them into the ByteArray, and calls ksproxy.ax->Kernelbase.dll->ntdll.dll. The last disassembly I can see in user mode, is 

mov     r10,rcx
mov     eax,47h
syscall
ret

With step-by-step debugger, I can easily see that the ByteArray is constructed exactly as it is supposed to be, according to the document. But I cannot find the usbdevfs_ctrltransfer structure, or its Windows equivalent.

Specifically, we suspect that the value of wIndex, specified in the document, applies to an older version of hardware, and that the Windows DLL actually uses 0x0400 instead of 0x0402.

Any hint (including hardware or software USB sniffers, emulators, etc.) how we can try to verify this unsigned short will be greatly appreciated.

Update

Reading https://reverseengineering.stackexchange.com/questions/2416/how-to-reverse-engineer-simple-usb-device-windows-linux and https://reverseengineering.stackexchange.com/questions/1786/usb-dongle-traffic-monitoring. It looks like these tools are not compatible with Windows 8.1 x64.

Community
  • 1
  • 1
Alex Cohn
  • 56,089
  • 9
  • 113
  • 307
  • I prefer this USB hardware analyzer for sniffing packets, [Ellisys USB Explorer 200](http://www.ellisys.com/products/usbex200/), add on decoding and you have yourself a nice tool for reverse engineering and debugging USB easily. – Preston May 26 '14 at 02:48
  • I like the [Beagle 12](http://www.totalphase.com/products/beagle-usb12/). – David Grayson May 29 '14 at 04:36
  • The USB device filesystem (USBDEVFS) is a Linux thing so I don't understand why you are even talking about it for a piece of Windows software. If this device you have uses winusb.sys as the driver, then the 3rd-party DLL should be making calls to setupapi.dll and winusb.dll. – David Grayson May 29 '14 at 04:37
  • @DavidGrayson: I don't see calls to `setupapi.dll` and `winusb.dll` in my debugger; instead, I can trace `ksproxy.ax`->`Kernelbase.dll`->`ntdll.dll`. You are right that USBEVFS is not for Windows. But I expected there to be a Windows equivalent for Linux `USBDEVFS_CONTROL` ioctl (`0xC0185500`). – Alex Cohn May 29 '14 at 09:38
  • 1
    If you look at the properties of this thing in the Device Manager, what drivers do you see in the list? Does it say winusb.sys or something else? – David Grayson May 29 '14 at 15:04
  • @DavidGrayson, it doesn't look even similar to what you describe. When I connect the device to my laptop, I see a new `USB Camera` under *Imaging devices*, which is totally expected, because the device is a camera. I also find a new `USB Composite Device`, `USB Printing Support`, and `No Printer Attached` -- all three under *USB controllers*. None lists `setupapi.dll` or `winusb.dll`. I would be very much obliged if you have any hint on how this is related to `USBDEVFS_CONTROL` ioctl. – Alex Cohn Jun 01 '14 at 09:56

1 Answers1

0

While working on the Xbox OS and peripherals, we always used the CATC Chief USB capture hardware, which works as a man-in-the-middle device (it looks like it has been superseded by the Teledyne LeCroy protocol analyzers).

The traffic capture capabilities were indispensable in diagnosing hardware and software errors (bulk, HID, isoch).

Sample capture view (from the manual):

USB Capture Trace

josh poley
  • 7,236
  • 1
  • 25
  • 25