2

In my app I use a RichEdit control (RICHED20.DLL) with syntax highlighting implemented on top of it. Quite a lot of code was necessary to make this work correctly. Internally, everything is stored as RTF and reformatted on the fly. This code has been working stable for over 10 years now in lots of different environments.

However, recently one user has managed to make my app freeze. He even managed to come up with a description of steps that can be used to clearly reproduce the freeze. Thus, I was able to debug it with WinDbg and got the report that it hangs in CTxtEdit::TxSendMessage. This is what WinDbg says when I break the program as soon as it hangs and do a !analyze -hang in WinDbg:

Probably caused by : RICHED20.DLL ( riched20!CTxtEdit::TxSendMessage+132e )

So does this look like a bug in the RichEdit control? Is there anything else I could debug here? I don't really know where to look because the hang is apparently not in my app. I've already checked and made sure that neither the WndProc of my RichEdit control nor the WndProc of the top-level receive any messages during the hang. It really seems to hang in RICHED20.DLL so to me it looks like a RichEdit control bug.

Could that be the case or what else could I try to debug the issue further? I'm out of ideas here.

Andreas
  • 9,245
  • 9
  • 49
  • 97
  • need more info - full stack trace dump how minimum, etc. or, the best have binary app and description of steps that can be used to clearly reproduce the freeze – RbMm Jun 04 '21 at 19:50
  • @RbMm: Here's a test binary: http://www.softwarefailure.de/tmp/rbmmtest.zip The instructions on how to reproduce the hangup are in the file `readme.txt`. Thanks for your help! – Andreas Jun 04 '21 at 20:02
  • 1
    yes, i view. this is infinite loop and hung (thread running all time). really error in *RICHED20.DLL*. call stack is https://pastebin.com/ucntYytx need some time for look in more detail, where error – RbMm Jun 04 '21 at 20:13
  • 1
    error exactly in function - `long CTxtPtr::FindEOP(long,long *)` - code never exit from it. infinit spin here, calling `unsigned short const * CTxtPtr::GetPchReverse(long &,long *)`. this not your error, but *RICHED20.DLL* – RbMm Jun 04 '21 at 20:16
  • Thanks for checking! Anything that I can do about this? – Andreas Jun 04 '21 at 20:17
  • i need bit more time fore research in detail - why richedit hang here and are possible something do.. need undertand it algorithm, where error. think this is very rare error, if still exist (nobody yet note it) and faster of all you only can not use *Riched20.dll* (it very old) but *Msftedit.dll* - [Versions of Rich Edit](https://learn.microsoft.com/en-us/windows/win32/controls/about-rich-edit-controls#versions-of-rich-edit) - try migrate to it and look - are error still exist – RbMm Jun 04 '21 at 20:24
  • I tried `Msftedit.dll` but it doesn't seem to be compatible with `Riched20.dll`. When I use `RICHEDIT50` instead of `RICHEDIT20W` in `CreateWindow` my app doesn't work at all and I don't want to rewrite everything. The code for syntax highlighting is very complicated. – Andreas Jun 04 '21 at 20:31
  • 1
    i research in detail. inside function `FindEOP` (it called as `FindEOP(cchMax = tomBackward, 0)`) exist loop `while(cchMax > 0)` and by mistake - function never exit from this loop. the `cchMax` all time is have value `tomForward = -tomBackward` which is > 0 and not updated inside loop. you can not fix this by self. but i open your document in *wordpad.exe* which is used *Msftedit.dll* and here no bug – RbMm Jun 04 '21 at 21:58
  • 1
    @Andreas: it's very easy to replace it. 1) Change the window class for the richedit control in your dialog resource to `RichEdit50W`, or create it using that class name. 2) Call `LoadLibrary(L"msftedit.dll");` somewhere during your process initialization before the window with the richedit is shown. The rest should be backwards compatible. – c00000fd Jun 05 '21 at 04:45
  • @c00000fd: Right, the problems with `RICHEDIT50` were caused because I didn't load `msftedit.dll`. When doing that, it works. Interestingly, the hang still shows up with `RICHEDIT50` on Windows 7 but not on Windows 10. So it seems to have been fixed some time after Windows 7. – Andreas Jun 05 '21 at 10:22
  • @Andreas: yes, it's a bug in Windows like rbmm has found. – c00000fd Jun 05 '21 at 15:56

0 Answers0