FreeLibrary, madExcept and win8

delphi package - automated exception handling
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

FreeLibrary, madExcept and win8

Post by SørenKann »

Howdy,

I have condensed a problem we have experienced [3] to a main program [1] loading and unloading a dll including madExcept [2]

1. When I run it on win 8.1, the dll is not removed from memory when unloaded (FreeLibrary) - verified using Process Explorer
2. When I run it on win 7, it works
3. If I remove madExcept from the dll, it works on win 8.1
4. If I keep madExcept in the dll, but remove my post-build action, calling madExceptPatch to include the map-file in the dll, it works on win 8.1

My guess is the madExceptPatch does something the dll-file, that win 8.1 does not like.

Any ideas on what to do?

br Søren Kann, Widex A/S

PS the problem occurs on Delphi XE end 10, and on madExcept 4.0.14 and earlier

[1] Main Program
================
program Project1;

{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, Windows;

var handle, i: cardinal; res: boolean;
begin
for i := 0 to 2 do begin
WriteLn('LoadLibrary?'); ReadLn;
handle := LoadLibrary(PWideChar('Tynd.dll')) ;
if handle=0 then WriteLn('*** Load failed ***');

WriteLn('FreeLibrary?'); ReadLn;
if handle<>0 then res := FreeLibrary(handle);
if not res then WriteLn('*** Free failed ***');
end;
end.

[2] Dll
=======
library Tynd;

uses madExcept, Windows;
{$R *.res}

procedure DllMain(reason: integer); begin end;

begin
DllProc := @DllMain;
DllProc(DLL_PROCESS_ATTACH);
end.

[3] The original problem
========================
We have a rather large dll, which uses madExcept. To test this dll, we use DUnit2, and the main setup/teardown of the testcases loads/unloads the dll. Until we left windows 7, this just worked. Now, on windows 8.1, the dll does not unload: that is, the dll-main is called with DLL_PROCESS_DETACH, but the image is not removed from memory, which causes the test application to run out of memory.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Post by madshi »

I've just tried to reproduce this on my WIndows 8.1 x64 development machine with Delphi Seattle 10 and madExcept 4.0.14, and it seems to work just fine. Here's my test project with full sources and compiled files:

http://madshi.net/DllTest.rar

Does this project (as compiled by me) work for you? If it does, try to recompile both DLL and EXE. Does it still work?
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Post by SørenKann »

When I run the Project1.exe generated by you, I end up with 10 instances of process2.dll in my memory image
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Post by madshi »

What is a "memory image"?

Windows doesn't even support loading the same DLL file multiple times in the same process!

What is the output of my test project on your PC?
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Post by SørenKann »

I use "Process Explorer" to show which dll's are loaded by an application. After I have run your Project1.exe, the picture is:
Attachments
Project1.png
Project1.png (113.51 KiB) Viewed 13767 times
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Post by SørenKann »

And the last part of the console log is:
Attachments
console.png
console.png (10.71 KiB) Viewed 13767 times
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Post by madshi »

Ah, I see, thanks.

The cause of the problem is that madExcept itself is multi-threaded. It creates a secondary thread which does all the exception handling, and that thread keeps running until madExcept is finalized. Finalization only occurs when the dll is unloaded, so it happens during DllMain(PROCESS_DETACH). At that moment madExcept cannot really successfully ask the thread to close down cleanly, so madExcept has no other choice than to terminate the exception helper thread... :( This seemed to work well enough in Windows 7, but it seems Windows 8.1 doesn't like it, for some reason.

Is it your own EXE we're talking about here? Do you have prior notice before your dll gets unloaded?

madExcept.pas exports a function named "FinalizeMadExcept", which you can manually call. E.g. you could do "exports FinalizeMadExcept", and then your EXE could use GetProcAddress(yourDll, "FinalizeMadExcept") and then call that API (no parameters) before actually unloading the dll.

It's all a bit ugly, but right now I don't have a better solution, unfortunately. Maybe at some point I need to redesign madExcept (at least when used in a dll) to not create secondary threads unless absolutely necessary.

One other alternative would be to uncheck "link in madExcept code" in your DLL project. If you do that, the map information is still added, so if your EXE is your own, and is also compiled with madExcept, it should be able to handle most of the DLL exceptions, too. However, crashes in DllMain (and unit initialization/finalization etc) would not be caught that way. You'd probably see "runtime error 216" messages for those.

If the EXE is not your own, we have a problem.
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Post by SørenKann »

Thanks
I will be back when I have had time to follow your directions.
SørenKann
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Post by SørenKann »

Bingo :greenBalloon:
I added the FinalizeMadExcept-call in the DUnit2 test-teardown just before unloading the dll, and now I can run a suite of 357 test-cases on win 8 as well as on win 7.
Thanks for the support.
fpiette
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm
Contact:

Re: FreeLibrary, madExcept and win8

Post by fpiette »

Hello Mathias,

I have the same issue but with Windows 10 (1909 18363.535, x64).
I tried the fix you suggested but it doesn't work. FreeLibrary hangs the application (32 bit).
Here is what I have doen:
1) In the DLL, added :

Code: Select all

       exports FinalizeMadExcept;
2) Recompiled the DLL
3) In the calling application, I added some code right before FreeLibrary:

Code: Select all

    for AddOnIndex := Low(FAddOnInfo) to High(FAddOnInfo) do begin
        if FAddOnInfo[AddOnIndex].DllHandle <> 0 then begin
            @MadExceptFinalize := GetProcAddress(FAddOnInfo[AddOnIndex].DllHandle, 'FinalizeMadExcept');   //<====   ADDED
            if @MadExceptFinalize <> nil then   //<====   ADDED
                MadExceptFinalize();   //<====   ADDED
            FreeLibrary(FAddOnInfo[AddOnIndex].DllHandle);
            FAddOnInfo[AddOnIndex].DllHandle := 0;
        end;
    end;
I have the additional declarations:

Code: Select all

   type TMadExceptFinalize = procedure;
   var  MadExceptFinalize : TMadExceptFinalize;
4) Recompiled the calling application
5) Run the application, it works as expected
6) Close the application: it hangs.
===> Using the debugger, I see that FreeLibrary never returns.

Removing madExcept from calling application: works.
Removing madExcept from dll: works.
Using madExcept in both DLL and calling application: fails

Any solution or workaround available?
Thanks
zunzster
Posts: 58
Joined: Wed Oct 29, 2008 3:43 am

Re: FreeLibrary, madExcept and win8

Post by zunzster »

This DLL helper thread shutdown dilemma is one I've seen Raymond Chen of Microsoft blog about a number of times.

https://devblogs.microsoft.com/oldnewthing/?p=19233
https://devblogs.microsoft.com/oldnewthing/?p=99675
https://devblogs.microsoft.com/oldnewthing/?p=2733
fpiette
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm
Contact:

Re: FreeLibrary, madExcept and win8

Post by fpiette »

Thanks Zunster. Those articles are very interesting, especialy the last one. But what is exposed is not for me, the application writer, but for Madshi to revise madExcept code. Madshi, are you listening?

Regards,
F. Piette
Embarcadero MVP
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Post by madshi »

Do you have freeze checking activated in the DLL? If so, please try deactivating it. Does that help?
fpiette
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm
Contact:

Re: FreeLibrary, madExcept and win8

Post by fpiette »

No freeze checking activated in the DLL nor in the application.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Post by madshi »

Can you reproduce the problem in a "simple" test project?
Post Reply