FreeLibrary, madExcept and win8

delphi package - automated exception handling

FreeLibrary, madExcept and win8

Postby SørenKann » Mon Apr 11, 2016 9:17 am

Howdy,

I have condensed a problem we have experienced [3] to a main program [1] loading and unloading a dll including madExcept [2]

1. When I run it on win 8.1, the dll is not removed from memory when unloaded (FreeLibrary) - verified using Process Explorer
2. When I run it on win 7, it works
3. If I remove madExcept from the dll, it works on win 8.1
4. If I keep madExcept in the dll, but remove my post-build action, calling madExceptPatch to include the map-file in the dll, it works on win 8.1

My guess is the madExceptPatch does something the dll-file, that win 8.1 does not like.

Any ideas on what to do?

br Søren Kann, Widex A/S

PS the problem occurs on Delphi XE end 10, and on madExcept 4.0.14 and earlier

[1] Main Program
================
program Project1;

{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, Windows;

var handle, i: cardinal; res: boolean;
begin
for i := 0 to 2 do begin
WriteLn('LoadLibrary?'); ReadLn;
handle := LoadLibrary(PWideChar('Tynd.dll')) ;
if handle=0 then WriteLn('*** Load failed ***');

WriteLn('FreeLibrary?'); ReadLn;
if handle<>0 then res := FreeLibrary(handle);
if not res then WriteLn('*** Free failed ***');
end;
end.

[2] Dll
=======
library Tynd;

uses madExcept, Windows;
{$R *.res}

procedure DllMain(reason: integer); begin end;

begin
DllProc := @DllMain;
DllProc(DLL_PROCESS_ATTACH);
end.

[3] The original problem
========================
We have a rather large dll, which uses madExcept. To test this dll, we use DUnit2, and the main setup/teardown of the testcases loads/unloads the dll. Until we left windows 7, this just worked. Now, on windows 8.1, the dll does not unload: that is, the dll-main is called with DLL_PROCESS_DETACH, but the image is not removed from memory, which causes the test application to run out of memory.
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby madshi » Mon Apr 11, 2016 12:18 pm

I've just tried to reproduce this on my WIndows 8.1 x64 development machine with Delphi Seattle 10 and madExcept 4.0.14, and it seems to work just fine. Here's my test project with full sources and compiled files:

http://madshi.net/DllTest.rar

Does this project (as compiled by me) work for you? If it does, try to recompile both DLL and EXE. Does it still work?
madshi
Site Admin
 
Posts: 10207
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Postby SørenKann » Mon Apr 11, 2016 12:30 pm

When I run the Project1.exe generated by you, I end up with 10 instances of process2.dll in my memory image
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby madshi » Mon Apr 11, 2016 1:11 pm

What is a "memory image"?

Windows doesn't even support loading the same DLL file multiple times in the same process!

What is the output of my test project on your PC?
madshi
Site Admin
 
Posts: 10207
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Postby SørenKann » Mon Apr 11, 2016 1:15 pm

I use "Process Explorer" to show which dll's are loaded by an application. After I have run your Project1.exe, the picture is:
Attachments
Project1.png
Project1.png (113.98 KiB) Viewed 5248 times
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby SørenKann » Mon Apr 11, 2016 1:18 pm

And the last part of the console log is:
Attachments
console.png
console.png (10.78 KiB) Viewed 5248 times
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby madshi » Mon Apr 11, 2016 2:15 pm

Ah, I see, thanks.

The cause of the problem is that madExcept itself is multi-threaded. It creates a secondary thread which does all the exception handling, and that thread keeps running until madExcept is finalized. Finalization only occurs when the dll is unloaded, so it happens during DllMain(PROCESS_DETACH). At that moment madExcept cannot really successfully ask the thread to close down cleanly, so madExcept has no other choice than to terminate the exception helper thread... :( This seemed to work well enough in Windows 7, but it seems Windows 8.1 doesn't like it, for some reason.

Is it your own EXE we're talking about here? Do you have prior notice before your dll gets unloaded?

madExcept.pas exports a function named "FinalizeMadExcept", which you can manually call. E.g. you could do "exports FinalizeMadExcept", and then your EXE could use GetProcAddress(yourDll, "FinalizeMadExcept") and then call that API (no parameters) before actually unloading the dll.

It's all a bit ugly, but right now I don't have a better solution, unfortunately. Maybe at some point I need to redesign madExcept (at least when used in a dll) to not create secondary threads unless absolutely necessary.

One other alternative would be to uncheck "link in madExcept code" in your DLL project. If you do that, the map information is still added, so if your EXE is your own, and is also compiled with madExcept, it should be able to handle most of the DLL exceptions, too. However, crashes in DllMain (and unit initialization/finalization etc) would not be caught that way. You'd probably see "runtime error 216" messages for those.

If the EXE is not your own, we have a problem.
madshi
Site Admin
 
Posts: 10207
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Postby SørenKann » Tue Apr 12, 2016 8:11 am

Thanks
I will be back when I have had time to follow your directions.
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby SørenKann » Tue Apr 12, 2016 1:27 pm

Bingo :greenBalloon:
I added the FinalizeMadExcept-call in the DUnit2 test-teardown just before unloading the dll, and now I can run a suite of 357 test-cases on win 8 as well as on win 7.
Thanks for the support.
SørenKann
 
Posts: 6
Joined: Mon Apr 11, 2016 8:56 am

Re: FreeLibrary, madExcept and win8

Postby fpiette » Tue Jan 07, 2020 1:58 pm

Hello Mathias,

I have the same issue but with Windows 10 (1909 18363.535, x64).
I tried the fix you suggested but it doesn't work. FreeLibrary hangs the application (32 bit).
Here is what I have doen:
1) In the DLL, added :
Code: Select all
       exports FinalizeMadExcept;

2) Recompiled the DLL
3) In the calling application, I added some code right before FreeLibrary:
Code: Select all
    for AddOnIndex := Low(FAddOnInfo) to High(FAddOnInfo) do begin
        if FAddOnInfo[AddOnIndex].DllHandle <> 0 then begin
            @MadExceptFinalize := GetProcAddress(FAddOnInfo[AddOnIndex].DllHandle, 'FinalizeMadExcept');   //<====   ADDED
            if @MadExceptFinalize <> nil then   //<====   ADDED
                MadExceptFinalize();   //<====   ADDED
            FreeLibrary(FAddOnInfo[AddOnIndex].DllHandle);
            FAddOnInfo[AddOnIndex].DllHandle := 0;
        end;
    end;

I have the additional declarations:
Code: Select all
   type TMadExceptFinalize = procedure;
   var  MadExceptFinalize : TMadExceptFinalize;

4) Recompiled the calling application
5) Run the application, it works as expected
6) Close the application: it hangs.
===> Using the debugger, I see that FreeLibrary never returns.

Removing madExcept from calling application: works.
Removing madExcept from dll: works.
Using madExcept in both DLL and calling application: fails

Any solution or workaround available?
Thanks
fpiette
 
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm

Re: FreeLibrary, madExcept and win8

Postby zunzster » Tue Jan 07, 2020 9:34 pm

This DLL helper thread shutdown dilemma is one I've seen Raymond Chen of Microsoft blog about a number of times.

https://devblogs.microsoft.com/oldnewthing/?p=19233
https://devblogs.microsoft.com/oldnewthing/?p=99675
https://devblogs.microsoft.com/oldnewthing/?p=2733
zunzster
 
Posts: 56
Joined: Wed Oct 29, 2008 3:43 am

Re: FreeLibrary, madExcept and win8

Postby fpiette » Wed Jan 08, 2020 7:19 am

Thanks Zunster. Those articles are very interesting, especialy the last one. But what is exposed is not for me, the application writer, but for Madshi to revise madExcept code. Madshi, are you listening?

Regards,
F. Piette
Embarcadero MVP
fpiette
 
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm

Re: FreeLibrary, madExcept and win8

Postby madshi » Wed Jan 08, 2020 2:46 pm

Do you have freeze checking activated in the DLL? If so, please try deactivating it. Does that help?
madshi
Site Admin
 
Posts: 10207
Joined: Sun Mar 21, 2004 5:25 pm

Re: FreeLibrary, madExcept and win8

Postby fpiette » Thu Jan 09, 2020 8:09 am

No freeze checking activated in the DLL nor in the application.
fpiette
 
Posts: 7
Joined: Tue Jan 07, 2020 1:47 pm

Re: FreeLibrary, madExcept and win8

Postby madshi » Thu Jan 09, 2020 1:54 pm

Can you reproduce the problem in a "simple" test project?
madshi
Site Admin
 
Posts: 10207
Joined: Sun Mar 21, 2004 5:25 pm

Next

Return to madExcept

Who is online

Users browsing this forum: Google [Bot] and 33 guests

cron