Memory leaks reported on exceptions and typeid

delphi package - automated exception handling
Post Reply
rsassen
Posts: 24
Joined: Fri Jun 20, 2014 2:41 pm

Memory leaks reported on exceptions and typeid

Post by rsassen »

Hi,

When I create a VCL project (C++-Builder XE7 update 1) with MadExcept and memory leak reporting enabled, every time an exception is handled or when type information is requested it results in a memory leak reported by MadExcept. When I use Codeguard instead of MadExcept it does not report these leaks. (I'm compiling in Debug mode).

I'm trying to figure out if these are real leaks, or not. A lot of these leaks a reported for my project, and it not only takes quite some time to generate the leak report each time I close the application, but they also obscure the leaks in my own code I want to find.

For example a new VCL forms application with just a button and a onClick handler with the code below result in three reported leaks when the button is pressed once:

Code: Select all

   String name = typeid(int).name();

   try
   {
      throw Exception("Oops");
   }
   catch(const Exception& E)
   {
   }
Leak report:

Code: Select all

allocation number: 2075
program up time: 4,08 s
type: GetMem
address: $24d4cd8
size: 28
access rights: read/write

main thread ($89c):
671cc9f6 madExcept32.dll madExceptDbg    1575 GetMemCallback
3207ddeb CC32160MT.DLL                        _malloc
320705d4 CC32160MT.DLL                        $bnew
320804d6 CC32160MT.DLL                        ___VCL_add_EH
320ada62 CC32160MT.DLL                        ____ExceptionHandler
00404d6c Project2.exe                         __ExceptionHandler
7794015e ntdll.dll                            KiUserExceptionDispatcher
320ad3fc CC32160MT.DLL                        _ThrowExceptionLDTC
00403a6c Project2.exe    Unit2.cpp         25 TForm2.Button1Click
505c3183 vcl210.bpl      Vcl.Controls    7348 TControl.Click
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
505c7847 vcl210.bpl      Vcl.Controls   10107 DoControlMsg
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
5070ce98 vcl210.bpl      Vcl.Forms       4427 TCustomForm.WndProc
505c6d1c vcl210.bpl      Vcl.Controls    9750 TWinControl.MainWndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f96d0 USER32.dll                           SendMessageW
75300d58 USER32.dll                           CallWindowProcW
505c77f2 vcl210.bpl      Vcl.Controls   10079 TWinControl.DefaultHandler
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f7895 USER32.dll                           DispatchMessageW
5071635b vcl210.bpl      Vcl.Forms      10352 TApplication.ProcessMessage

memory dump: 
024d4cd8  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
024d4ce8  00 00 00 00 00 00 00 00 - 00 00 00 00              ............

allocation number: 2067
program up time: 4,05 s
type: GetMem
address: $24dc0a0
size: 28
access rights: read/write

main thread ($89c):
671cc9f6 madExcept32.dll madExceptDbg    1575 GetMemCallback
3207ddeb CC32160MT.DLL                        _malloc
320705d4 CC32160MT.DLL                        $bnew
320804d6 CC32160MT.DLL                        ___VCL_add_EH
320ad3fc CC32160MT.DLL                        _ThrowExceptionLDTC
00403a6c Project2.exe    Unit2.cpp         25 TForm2.Button1Click
505c3183 vcl210.bpl      Vcl.Controls    7348 TControl.Click
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
505c7847 vcl210.bpl      Vcl.Controls   10107 DoControlMsg
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
5070ce98 vcl210.bpl      Vcl.Forms       4427 TCustomForm.WndProc
505c6d1c vcl210.bpl      Vcl.Controls    9750 TWinControl.MainWndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f96d0 USER32.dll                           SendMessageW
75300d58 USER32.dll                           CallWindowProcW
505c77f2 vcl210.bpl      Vcl.Controls   10079 TWinControl.DefaultHandler
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f7895 USER32.dll                           DispatchMessageW
5071635b vcl210.bpl      Vcl.Forms      10352 TApplication.ProcessMessage

memory dump: 
024dc0a0  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
024dc0b0  00 00 00 00 00 00 00 00 - 00 00 00 00              ............

allocation number: 2063
program up time: 1,42 s
type: GetMem
address: $250d378
size: 20
access rights: read/write

main thread ($89c):
671cc9f6 madExcept32.dll madExceptDbg    1575 GetMemCallback
3207ddeb CC32160MT.DLL                        _malloc
320705d4 CC32160MT.DLL                        $bnew
320ae6f0 CC32160MT.DLL                        __GetTypeInfo
00403a00 Project2.exe    Unit2.cpp         21 TForm2.Button1Click
505c3183 vcl210.bpl      Vcl.Controls    7348 TControl.Click
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
505c7847 vcl210.bpl      Vcl.Controls   10107 DoControlMsg
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
5070ce98 vcl210.bpl      Vcl.Forms       4427 TCustomForm.WndProc
505c6d1c vcl210.bpl      Vcl.Controls    9750 TWinControl.MainWndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f96d0 USER32.dll                           SendMessageW
75300d58 USER32.dll                           CallWindowProcW
505c77f2 vcl210.bpl      Vcl.Controls   10079 TWinControl.DefaultHandler
505c76e2 vcl210.bpl      Vcl.Controls   10038 TWinControl.WndProc
505e7c20 vcl210.bpl      Vcl.StdCtrls    5163 TButtonControl.WndProc
5016e218 rtl210.bpl      System.Classes 16600 StdWndProc
752f7895 USER32.dll                           DispatchMessageW
5071635b vcl210.bpl      Vcl.Forms      10352 TApplication.ProcessMessage

memory dump: 
0250d378  90 4e 0d 32 a4 3a 40 00 - 00 00 00 00 00 00 00 00  .N.2.:@.........
0250d388  00 00 00 00                                        ....

I've browsed the Embarcadero source code a bit, and I found some code there that suggests at least the GetTypeInfo leak should not happen.

For example in "C:\Program Files (x86)\Embarcadero\Studio\15.0\source\cpprtl\Source\except\xxtype.cpp" I find the FreeHashTab function:

Code: Select all

// Free the hash memory so that CodeGuard and MemorySleuth don't have a cow.
static void FreeHashTab (void)
{
#pragma exit FreeHashTab 1  /* Finalize the hash memory blocks */
                            /* This must happen before the heap shuts down */
...
If this is indeed the place where the type info hash memory is freed on application shutdown, why does MadExcept still report these leaks?

Can you reproduce these results?
Are they real leaks?
Do you have any idea why these leaks are reported by MadExcept and not by CodeGuard?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: Memory leaks reported on exceptions and typeid

Post by madshi »

I've looked through the BCB source code to try to understand what happens there, but it's extremely hard to understand. E.g. there's this "FreeHashTag". But is it actually called? And does it properly free everything? One big problem is that I don't know how to properly debug all this. E.g. how to set a breakpoint on FreeHashTag?

Generally, of course there's a chance that madExcept could have a bug which creates some false positives. However, in the recent past in most situations where users asked me to check if a leak is a real leak, it usually was. I'm not sure if that's the case here, too. Maybe yes, maybe not. If it's a real leak, why does CodeGuard not report it? There could be various reasons for that. E.g. if CodeGuard knows it's a leak in the RTL then maybe they simple filter it away.

In a future version I plan to offer some APIs to allow you to filter known leaks (which you can't do anything about) away by providing a leak stack trace or something like that.
rsassen
Posts: 24
Joined: Fri Jun 20, 2014 2:41 pm

Re: Memory leaks reported on exceptions and typeid

Post by rsassen »

Thank you for looking into it.

Unfortunately I don't know how to debug it either :?
What I understand from the documentation is that the "#pragma exit" causes the specified FreeHashTab function to be called when the program exits. Of course whether it is actually called remains a question.

I don't know when MadExcept starts the process of determining the memory leaks. FreeHashTab has priority 1 in the pragma exit, which is a very high priority and as such will be called very late in the process of terminating the process. Could it be that MadExcept reports the leaks before the process if completely terminated, i.e. before FreeHashTab is called?

Regarding the question of why CodeGuard does not report the leak, the remark in the code

Code: Select all

// Free the hash memory so that CodeGuard and MemorySleuth don't have a cow.
seems to suggest that the reason the FreeHashTab function is implemented is that CodeGuard does not filter the leak.

I very much like your proposal of implementing an API to filter known leaks.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: Memory leaks reported on exceptions and typeid

Post by madshi »

In theory leak checking should be done *very* late, after all the Delphi and BCB VCL and RTL finalization.
rsassen
Posts: 24
Joined: Fri Jun 20, 2014 2:41 pm

Re: Memory leaks reported on exceptions and typeid

Post by rsassen »

Then it seems I will have to wait for the new API to filter know leaks. :D

Again, thank you for looking into it!
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: Memory leaks reported on exceptions and typeid

Post by madshi »

Here's a new beta build:

http://madshi.net/madCollectionBeta.exe (installer 2.7.12.14)

You can now e.g. call:

Code: Select all

HideLeak(L"_malloc\\$bnew\\__GetTypeInfo");
Unfortunately filtering out leaks this way does not improve performance in any way. But I don't know a better way atm.
rsassen
Posts: 24
Joined: Fri Jun 20, 2014 2:41 pm

Re: Memory leaks reported on exceptions and typeid

Post by rsassen »

Wow, you have been busy :D

I've tried the beta build. The example code you provided did not work for me, but when I replaced the double backslashes with pipe symbols I got it working (your source code mentions this syntax). E.g.:

Code: Select all

HideLeak(L"_malloc|$bnew|___VCL_add_EH");
HideLeak(L"_malloc|$bnew|__GetTypeInfo");
HideLeak(L"_malloc|$bnew|$bnwa|type_info.name");
It's too bad it still takes quite some time for my application to process all these leak, but at least the leaks I'm interested in are not hidden in a forest of known leaks.

Thanks!
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: Memory leaks reported on exceptions and typeid

Post by madshi »

Ah, sorry about the double backslash, stupid mistake.

Yeah, it would be nicer if I wouldn't have to calculate the callstack for the leaks first. But unless we find a bug in the leak reporting and these turn out to be "untrue" leaks there's not much I can do. Well, maybe I could do a quick and stupid callstack first, and only if it looks to be a valid leak do a proper (slow) callstack? But that's not going to happen any time soon, I fear. Too busy with other stuff atm...
Post Reply