Page 1 of 2

FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 9:17 am
by SørenKann
Howdy,

I have condensed a problem we have experienced [3] to a main program [1] loading and unloading a dll including madExcept [2]

1. When I run it on win 8.1, the dll is not removed from memory when unloaded (FreeLibrary) - verified using Process Explorer
2. When I run it on win 7, it works
3. If I remove madExcept from the dll, it works on win 8.1
4. If I keep madExcept in the dll, but remove my post-build action, calling madExceptPatch to include the map-file in the dll, it works on win 8.1

My guess is the madExceptPatch does something the dll-file, that win 8.1 does not like.

Any ideas on what to do?

br Søren Kann, Widex A/S

PS the problem occurs on Delphi XE end 10, and on madExcept 4.0.14 and earlier

[1] Main Program
================
program Project1;

{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, Windows;

var handle, i: cardinal; res: boolean;
begin
for i := 0 to 2 do begin
WriteLn('LoadLibrary?'); ReadLn;
handle := LoadLibrary(PWideChar('Tynd.dll')) ;
if handle=0 then WriteLn('*** Load failed ***');

WriteLn('FreeLibrary?'); ReadLn;
if handle<>0 then res := FreeLibrary(handle);
if not res then WriteLn('*** Free failed ***');
end;
end.

[2] Dll
=======
library Tynd;

uses madExcept, Windows;
{$R *.res}

procedure DllMain(reason: integer); begin end;

begin
DllProc := @DllMain;
DllProc(DLL_PROCESS_ATTACH);
end.

[3] The original problem
========================
We have a rather large dll, which uses madExcept. To test this dll, we use DUnit2, and the main setup/teardown of the testcases loads/unloads the dll. Until we left windows 7, this just worked. Now, on windows 8.1, the dll does not unload: that is, the dll-main is called with DLL_PROCESS_DETACH, but the image is not removed from memory, which causes the test application to run out of memory.

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 12:18 pm
by madshi
I've just tried to reproduce this on my WIndows 8.1 x64 development machine with Delphi Seattle 10 and madExcept 4.0.14, and it seems to work just fine. Here's my test project with full sources and compiled files:

http://madshi.net/DllTest.rar

Does this project (as compiled by me) work for you? If it does, try to recompile both DLL and EXE. Does it still work?

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 12:30 pm
by SørenKann
When I run the Project1.exe generated by you, I end up with 10 instances of process2.dll in my memory image

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 1:11 pm
by madshi
What is a "memory image"?

Windows doesn't even support loading the same DLL file multiple times in the same process!

What is the output of my test project on your PC?

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 1:15 pm
by SørenKann
I use "Process Explorer" to show which dll's are loaded by an application. After I have run your Project1.exe, the picture is:

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 1:18 pm
by SørenKann
And the last part of the console log is:

Re: FreeLibrary, madExcept and win8

PostPosted: Mon Apr 11, 2016 2:15 pm
by madshi
Ah, I see, thanks.

The cause of the problem is that madExcept itself is multi-threaded. It creates a secondary thread which does all the exception handling, and that thread keeps running until madExcept is finalized. Finalization only occurs when the dll is unloaded, so it happens during DllMain(PROCESS_DETACH). At that moment madExcept cannot really successfully ask the thread to close down cleanly, so madExcept has no other choice than to terminate the exception helper thread... :( This seemed to work well enough in Windows 7, but it seems Windows 8.1 doesn't like it, for some reason.

Is it your own EXE we're talking about here? Do you have prior notice before your dll gets unloaded?

madExcept.pas exports a function named "FinalizeMadExcept", which you can manually call. E.g. you could do "exports FinalizeMadExcept", and then your EXE could use GetProcAddress(yourDll, "FinalizeMadExcept") and then call that API (no parameters) before actually unloading the dll.

It's all a bit ugly, but right now I don't have a better solution, unfortunately. Maybe at some point I need to redesign madExcept (at least when used in a dll) to not create secondary threads unless absolutely necessary.

One other alternative would be to uncheck "link in madExcept code" in your DLL project. If you do that, the map information is still added, so if your EXE is your own, and is also compiled with madExcept, it should be able to handle most of the DLL exceptions, too. However, crashes in DllMain (and unit initialization/finalization etc) would not be caught that way. You'd probably see "runtime error 216" messages for those.

If the EXE is not your own, we have a problem.

Re: FreeLibrary, madExcept and win8

PostPosted: Tue Apr 12, 2016 8:11 am
by SørenKann
Thanks
I will be back when I have had time to follow your directions.

Re: FreeLibrary, madExcept and win8

PostPosted: Tue Apr 12, 2016 1:27 pm
by SørenKann
Bingo :greenBalloon:
I added the FinalizeMadExcept-call in the DUnit2 test-teardown just before unloading the dll, and now I can run a suite of 357 test-cases on win 8 as well as on win 7.
Thanks for the support.

Re: FreeLibrary, madExcept and win8

PostPosted: Tue Jan 07, 2020 1:58 pm
by fpiette
Hello Mathias,

I have the same issue but with Windows 10 (1909 18363.535, x64).
I tried the fix you suggested but it doesn't work. FreeLibrary hangs the application (32 bit).
Here is what I have doen:
1) In the DLL, added :
Code: Select all
       exports FinalizeMadExcept;

2) Recompiled the DLL
3) In the calling application, I added some code right before FreeLibrary:
Code: Select all
    for AddOnIndex := Low(FAddOnInfo) to High(FAddOnInfo) do begin
        if FAddOnInfo[AddOnIndex].DllHandle <> 0 then begin
            @MadExceptFinalize := GetProcAddress(FAddOnInfo[AddOnIndex].DllHandle, 'FinalizeMadExcept');   //<====   ADDED
            if @MadExceptFinalize <> nil then   //<====   ADDED
                MadExceptFinalize();   //<====   ADDED
            FreeLibrary(FAddOnInfo[AddOnIndex].DllHandle);
            FAddOnInfo[AddOnIndex].DllHandle := 0;
        end;
    end;

I have the additional declarations:
Code: Select all
   type TMadExceptFinalize = procedure;
   var  MadExceptFinalize : TMadExceptFinalize;

4) Recompiled the calling application
5) Run the application, it works as expected
6) Close the application: it hangs.
===> Using the debugger, I see that FreeLibrary never returns.

Removing madExcept from calling application: works.
Removing madExcept from dll: works.
Using madExcept in both DLL and calling application: fails

Any solution or workaround available?
Thanks

Re: FreeLibrary, madExcept and win8

PostPosted: Tue Jan 07, 2020 9:34 pm
by zunzster
This DLL helper thread shutdown dilemma is one I've seen Raymond Chen of Microsoft blog about a number of times.

https://devblogs.microsoft.com/oldnewthing/?p=19233
https://devblogs.microsoft.com/oldnewthing/?p=99675
https://devblogs.microsoft.com/oldnewthing/?p=2733

Re: FreeLibrary, madExcept and win8

PostPosted: Wed Jan 08, 2020 7:19 am
by fpiette
Thanks Zunster. Those articles are very interesting, especialy the last one. But what is exposed is not for me, the application writer, but for Madshi to revise madExcept code. Madshi, are you listening?

Regards,
F. Piette
Embarcadero MVP

Re: FreeLibrary, madExcept and win8

PostPosted: Wed Jan 08, 2020 2:46 pm
by madshi
Do you have freeze checking activated in the DLL? If so, please try deactivating it. Does that help?

Re: FreeLibrary, madExcept and win8

PostPosted: Thu Jan 09, 2020 8:09 am
by fpiette
No freeze checking activated in the DLL nor in the application.

Re: FreeLibrary, madExcept and win8

PostPosted: Thu Jan 09, 2020 1:54 pm
by madshi
Can you reproduce the problem in a "simple" test project?