I wrote all that stuff below over several hours as I was testing stuff, and I've since found the root cause. So here's a <TLDR>:
This almost ties everything together.
The short version is that there was a form that was not being freed.
But it's a little more nuanced. It's to do with AllocateHwnd/DeallocateHwnd routines and components that depend on them such as TTimer and TPopupMenu. The TPopupMenu area is what kept falling over. In TPopupList.Add, the first time it's called, AllocateHwnd is called (creating window "TPUtilWindow"). There is a counter being maintained, and TPopupList.Remove calls DeallocateHwnd if the counter is 0. The application creates a bunch of TPopupMenus when it starts, in various forms. The form that is being leaked also creates a bunch of those. So when the application was shutting down, the counter only got reduced to 18~ (popup menus remaining somewhere), never getting to 0 and never calling DeallocateHwnd (complete application shut down follows). I think it doesn't matter if the form that is being leaked didn't free those TPopupMenus or they were leaked by the application earlier on, the key thing is that DeallocateHwnd did not get called, leaving the handle for Windows to put messages in. And the way AllocateHwnd works (which I've read about earlier today), it does some sort of patching (in MakeObjectInstance) so the WndProc that Windows would call for the TPopupList, would have the matching method/object. For some reason, the AV happens is when madExcept is preparing the leak report and calls a GetWindowText on one of these window handles. This ends up in Delphi around here:
Code: Select all
procedure TPopupList.MainWndProc(var Message: TMessage);
begin
try
WndProc(Message); <<<<
except
Application.HandleException(Self);
end;
end;
So it appears the intended WndProc is called back, but it stops here and can't call the real TPopupList.WndProc().
Sometimes it also lands here:
Code: Select all
function StdWndProc(Window: HWND; Message, WParam: Longint;
LParam: Longint): Longint; stdcall; assembler;
asm
XOR EAX,EAX
PUSH EAX
PUSH LParam
PUSH WParam
PUSH Message
MOV EDX,ESP
MOV EAX,[ECX].Longint[4]
CALL [ECX].Pointer <<<<<<<<<<<<<<<<<<<<<
ADD ESP,12
POP EAX
end;
Not sure if this is the correct/patched WndProc, nor am I sure as to why it jumps between these 2 randomly. I think the CALL above would end up in TPopupList.MainWndProc(), but something appears to be corrupt at this point to proceed. I guess it's half corrupt when it gets into TPopupList, to explain it failing there. That's me guessing, trying to make sense of it. I checked that TPopupList doesn't get freed until Classes's finalization.
So it's a mixed bag: messages shouldn't be firing into that window handle; if they are, they shouldn't cause the AV (Delphi's special code should still be in place); messages only sent when madExcept's leak reporting is enabled. Perhaps madExcept could avoid using API's that would send a message to existing windows? Just tossing ideas around. Apparently AllocateHwnd is not thread safe (see
https://www.thedelphigeek.com/2007/06/a ... -safe.html).
In conclusion (for now
), I will see how things behave in Delphi 12, but for now, the form that leaked 70000 bits of memory is freed. Now madExcept leak report is showing me some 260 leaks to check out - back to where I was a few days ago
I wonder how things would behave if I did manually controlled runtime leak testing with madExcept (this was going to be one of my next steps). Unfortunately I've spent too long on this today, but I will set a little time aside to test that scenario out (and again, I'll come back to this thread to post if all of this would have worked in Delphi 12 and I would have seen the big form leak to begin with).
If any info is needed or anything to try out, it would be my pleasure.
Thank you Mathias for your attention here. Sorry I have subjected you to these really long posts
</TLDR>
>The original issue occurred without leak reporting, right? So adding leak reporting on top will just add more possible causes for trouble.
Before I started leak testing, there were no exceptions caught in the IDE (i.e. when "Stop on Delphi Exceptions is on"), nor any exceptions were caught by madExcept when enabled only for exception handling, not leak reporting. But when leak reporting is turned on in madExcept, the application crashes at the stage when the report appears to have finished generating but the little window is still showing in the bottom-right corner of the screen.
So it's an odd one because on one hand, the application is most certainly leaving something unfreed that manages to cause a crash when madExcept's leak reporting is turned on (and who knows, if it's indeed an issue on its own, it may well manifest in other circumstances without madExcept+leak reporting). But on the other hand, madExcept could perhaps be dealing with this a little better. After all, I am using it to find leaks, and instead, an exception is introduced that shows up in the IDE, but missed by madExcept, whose primary purpose is to catch unhandled exceptions, no?
I am sure there is a problem in the application's code, and I will find what it is soon enough and post my findings. My bet is that even though the problem will be something leaking in the application, madExcept leak report generation process (right toward the end of it) could be slightly improved to deal with this case (whatever it turns out to be) and make madExcept just that bit better
You'll be the judge of that; I have to find the source of the problem
---
Because the exception is in Classes.StdWndProc(), I suspect some form/window is not freed and this form somehow ends up receiving messages when madExcept finishes producing the leak report. Something either manually pumps the messages, does something like an Application.ProcessMessages, does a SendMessage; or the message pump shouldn't be running by this point. If it comes to it, I should be able to find out to which window the message belongs to when StdWndProc fires (might use hooks to trace creation/destruction of HWNDs or try something like Winspector Spy to see if it can find the handle + name and class). Starting to suspect this 40 DLLs third-party library isn't squeaky clean either. We will see.
Well I started looking into it a little bit more, just to draw a line for the time being.....
I've attached 2 screenshots from x32dbg when debugging this issue. madExcept is enabled, leak reporting enabled, crash on buffer overrun/underrun disabled.
"1a - madexcept32.dll is about to call GetWindowTextW (insta crash on buffer overflow or underrun is disabled).png"
"1b - GetWindowTextW appears to lead to a crash in StdWndProc.png"
Indeed it's madExcept's call to GetWindowTextW is what's resulting in the message WM_GETTEXT arriving in StdWndProc and lead to a crash with error code 0xc000041d. And it does look like there are a lot of leaks - bits of strings in the debugger are showing leak count in the range of 70000
>In addition to secondary threads still running, I'd suggest to check for:
>1) Forms being freed twice.
I wonder how to do that? It's rare that this happens and one hopes for an exception when it does.
>2) Buffer overruns.
>You can test 2) by activating madExcept's "instantly crash on buffer overrun" feature. However, this will once more eat tons of additional RAM, so this could cause issues on its own.
This doesn't appear to have made a difference, but see point 3 below.
3) I've also tested the other option, to crash on buffer underrun.
The report generation window's progress bar reached about 40% before a crash - but this time, madExcept popped up, showing that exception has occurred in madExcept32.dll. So this is something new.
Screenshot "2 - crash in madExcept32.dll.png" attached.
Noticed it says compiled with "Delphi 12" for madExcept32.dll - this is probably a coincidence, but I've installed Delphi 12 a couple of days ago. Also, some leaks are being shown by madExcept if I don't go into the trouble area, so this DLL works with Delphi 7 just fine, but thought I'll mention, just in case.
---
Other unrelated bits I've noticed - just some feedback/question. When doing a couple of these tests (buffer overrun and underrun) when expecting a crash, I didn't see madExcept popup, so I thought the settings didn't have any effect. I repeated the tests to be sure, and indeed madExcept popped up. I must've got distracted or something before, having not noticed madExcept's dialog? A bit later I noticed that madExcept appears to hide its exception dialog after a while. Is that so? I was taking screenshots of it for this post, and a couple of times the form just disappeared while I was print-screening/pasting. UPDATE: indeed when testing point 3 above, madExcept's dialog got closed after a few minutes - the application crashed with the same 2 Event Viewer entries as reported previously. Thought I'll mention this.
The other minor thing to consider: I use laptop for 2nd display which has 150% DPI settings (Win10). Moving madExcept's exception dialog to this laptop's screen makes it look a little odd: large non-client area (as per 150% DPI), while everything else looks teeny. I suspect adding some DPI awareness to madExcept's dialog would help those with super high res display and Windows set to increase text size.
Final nit from me: unimportant but maybe you feel like fixing: I managed to mess up the scrollbar in one of the madExcept dialogs (barely seen in one of the screenshots). I tried to make the window tall enough to screenshot all the listed threads, and tried to resize the window beyond the desktop's workspace area (under taskbar). Windows forced the form back into view, but madExcept dialog's scrollbars didn't adjust back.