FreeLibrary, madExcept and win8
FreeLibrary, madExcept and win8
Howdy,
I have condensed a problem we have experienced [3] to a main program [1] loading and unloading a dll including madExcept [2]
1. When I run it on win 8.1, the dll is not removed from memory when unloaded (FreeLibrary) - verified using Process Explorer
2. When I run it on win 7, it works
3. If I remove madExcept from the dll, it works on win 8.1
4. If I keep madExcept in the dll, but remove my post-build action, calling madExceptPatch to include the map-file in the dll, it works on win 8.1
My guess is the madExceptPatch does something the dll-file, that win 8.1 does not like.
Any ideas on what to do?
br Søren Kann, Widex A/S
PS the problem occurs on Delphi XE end 10, and on madExcept 4.0.14 and earlier
[1] Main Program
================
program Project1;
{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, Windows;
var handle, i: cardinal; res: boolean;
begin
for i := 0 to 2 do begin
WriteLn('LoadLibrary?'); ReadLn;
handle := LoadLibrary(PWideChar('Tynd.dll')) ;
if handle=0 then WriteLn('*** Load failed ***');
WriteLn('FreeLibrary?'); ReadLn;
if handle<>0 then res := FreeLibrary(handle);
if not res then WriteLn('*** Free failed ***');
end;
end.
[2] Dll
=======
library Tynd;
uses madExcept, Windows;
{$R *.res}
procedure DllMain(reason: integer); begin end;
begin
DllProc := @DllMain;
DllProc(DLL_PROCESS_ATTACH);
end.
[3] The original problem
========================
We have a rather large dll, which uses madExcept. To test this dll, we use DUnit2, and the main setup/teardown of the testcases loads/unloads the dll. Until we left windows 7, this just worked. Now, on windows 8.1, the dll does not unload: that is, the dll-main is called with DLL_PROCESS_DETACH, but the image is not removed from memory, which causes the test application to run out of memory.
I have condensed a problem we have experienced [3] to a main program [1] loading and unloading a dll including madExcept [2]
1. When I run it on win 8.1, the dll is not removed from memory when unloaded (FreeLibrary) - verified using Process Explorer
2. When I run it on win 7, it works
3. If I remove madExcept from the dll, it works on win 8.1
4. If I keep madExcept in the dll, but remove my post-build action, calling madExceptPatch to include the map-file in the dll, it works on win 8.1
My guess is the madExceptPatch does something the dll-file, that win 8.1 does not like.
Any ideas on what to do?
br Søren Kann, Widex A/S
PS the problem occurs on Delphi XE end 10, and on madExcept 4.0.14 and earlier
[1] Main Program
================
program Project1;
{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, Windows;
var handle, i: cardinal; res: boolean;
begin
for i := 0 to 2 do begin
WriteLn('LoadLibrary?'); ReadLn;
handle := LoadLibrary(PWideChar('Tynd.dll')) ;
if handle=0 then WriteLn('*** Load failed ***');
WriteLn('FreeLibrary?'); ReadLn;
if handle<>0 then res := FreeLibrary(handle);
if not res then WriteLn('*** Free failed ***');
end;
end.
[2] Dll
=======
library Tynd;
uses madExcept, Windows;
{$R *.res}
procedure DllMain(reason: integer); begin end;
begin
DllProc := @DllMain;
DllProc(DLL_PROCESS_ATTACH);
end.
[3] The original problem
========================
We have a rather large dll, which uses madExcept. To test this dll, we use DUnit2, and the main setup/teardown of the testcases loads/unloads the dll. Until we left windows 7, this just worked. Now, on windows 8.1, the dll does not unload: that is, the dll-main is called with DLL_PROCESS_DETACH, but the image is not removed from memory, which causes the test application to run out of memory.
Re: FreeLibrary, madExcept and win8
I've just tried to reproduce this on my WIndows 8.1 x64 development machine with Delphi Seattle 10 and madExcept 4.0.14, and it seems to work just fine. Here's my test project with full sources and compiled files:
http://madshi.net/DllTest.rar
Does this project (as compiled by me) work for you? If it does, try to recompile both DLL and EXE. Does it still work?
http://madshi.net/DllTest.rar
Does this project (as compiled by me) work for you? If it does, try to recompile both DLL and EXE. Does it still work?
Re: FreeLibrary, madExcept and win8
When I run the Project1.exe generated by you, I end up with 10 instances of process2.dll in my memory image
Re: FreeLibrary, madExcept and win8
What is a "memory image"?
Windows doesn't even support loading the same DLL file multiple times in the same process!
What is the output of my test project on your PC?
Windows doesn't even support loading the same DLL file multiple times in the same process!
What is the output of my test project on your PC?
Re: FreeLibrary, madExcept and win8
I use "Process Explorer" to show which dll's are loaded by an application. After I have run your Project1.exe, the picture is:
- Attachments
-
- Project1.png (113.51 KiB) Viewed 13945 times
Re: FreeLibrary, madExcept and win8
And the last part of the console log is:
- Attachments
-
- console.png (10.71 KiB) Viewed 13945 times
Re: FreeLibrary, madExcept and win8
Ah, I see, thanks.
The cause of the problem is that madExcept itself is multi-threaded. It creates a secondary thread which does all the exception handling, and that thread keeps running until madExcept is finalized. Finalization only occurs when the dll is unloaded, so it happens during DllMain(PROCESS_DETACH). At that moment madExcept cannot really successfully ask the thread to close down cleanly, so madExcept has no other choice than to terminate the exception helper thread... This seemed to work well enough in Windows 7, but it seems Windows 8.1 doesn't like it, for some reason.
Is it your own EXE we're talking about here? Do you have prior notice before your dll gets unloaded?
madExcept.pas exports a function named "FinalizeMadExcept", which you can manually call. E.g. you could do "exports FinalizeMadExcept", and then your EXE could use GetProcAddress(yourDll, "FinalizeMadExcept") and then call that API (no parameters) before actually unloading the dll.
It's all a bit ugly, but right now I don't have a better solution, unfortunately. Maybe at some point I need to redesign madExcept (at least when used in a dll) to not create secondary threads unless absolutely necessary.
One other alternative would be to uncheck "link in madExcept code" in your DLL project. If you do that, the map information is still added, so if your EXE is your own, and is also compiled with madExcept, it should be able to handle most of the DLL exceptions, too. However, crashes in DllMain (and unit initialization/finalization etc) would not be caught that way. You'd probably see "runtime error 216" messages for those.
If the EXE is not your own, we have a problem.
The cause of the problem is that madExcept itself is multi-threaded. It creates a secondary thread which does all the exception handling, and that thread keeps running until madExcept is finalized. Finalization only occurs when the dll is unloaded, so it happens during DllMain(PROCESS_DETACH). At that moment madExcept cannot really successfully ask the thread to close down cleanly, so madExcept has no other choice than to terminate the exception helper thread... This seemed to work well enough in Windows 7, but it seems Windows 8.1 doesn't like it, for some reason.
Is it your own EXE we're talking about here? Do you have prior notice before your dll gets unloaded?
madExcept.pas exports a function named "FinalizeMadExcept", which you can manually call. E.g. you could do "exports FinalizeMadExcept", and then your EXE could use GetProcAddress(yourDll, "FinalizeMadExcept") and then call that API (no parameters) before actually unloading the dll.
It's all a bit ugly, but right now I don't have a better solution, unfortunately. Maybe at some point I need to redesign madExcept (at least when used in a dll) to not create secondary threads unless absolutely necessary.
One other alternative would be to uncheck "link in madExcept code" in your DLL project. If you do that, the map information is still added, so if your EXE is your own, and is also compiled with madExcept, it should be able to handle most of the DLL exceptions, too. However, crashes in DllMain (and unit initialization/finalization etc) would not be caught that way. You'd probably see "runtime error 216" messages for those.
If the EXE is not your own, we have a problem.
Re: FreeLibrary, madExcept and win8
Thanks
I will be back when I have had time to follow your directions.
I will be back when I have had time to follow your directions.
Re: FreeLibrary, madExcept and win8
Bingo
I added the FinalizeMadExcept-call in the DUnit2 test-teardown just before unloading the dll, and now I can run a suite of 357 test-cases on win 8 as well as on win 7.
Thanks for the support.
I added the FinalizeMadExcept-call in the DUnit2 test-teardown just before unloading the dll, and now I can run a suite of 357 test-cases on win 8 as well as on win 7.
Thanks for the support.
Re: FreeLibrary, madExcept and win8
Hello Mathias,
I have the same issue but with Windows 10 (1909 18363.535, x64).
I tried the fix you suggested but it doesn't work. FreeLibrary hangs the application (32 bit).
Here is what I have doen:
1) In the DLL, added :
2) Recompiled the DLL
3) In the calling application, I added some code right before FreeLibrary:
I have the additional declarations:
4) Recompiled the calling application
5) Run the application, it works as expected
6) Close the application: it hangs.
===> Using the debugger, I see that FreeLibrary never returns.
Removing madExcept from calling application: works.
Removing madExcept from dll: works.
Using madExcept in both DLL and calling application: fails
Any solution or workaround available?
Thanks
I have the same issue but with Windows 10 (1909 18363.535, x64).
I tried the fix you suggested but it doesn't work. FreeLibrary hangs the application (32 bit).
Here is what I have doen:
1) In the DLL, added :
Code: Select all
exports FinalizeMadExcept;
3) In the calling application, I added some code right before FreeLibrary:
Code: Select all
for AddOnIndex := Low(FAddOnInfo) to High(FAddOnInfo) do begin
if FAddOnInfo[AddOnIndex].DllHandle <> 0 then begin
@MadExceptFinalize := GetProcAddress(FAddOnInfo[AddOnIndex].DllHandle, 'FinalizeMadExcept'); //<==== ADDED
if @MadExceptFinalize <> nil then //<==== ADDED
MadExceptFinalize(); //<==== ADDED
FreeLibrary(FAddOnInfo[AddOnIndex].DllHandle);
FAddOnInfo[AddOnIndex].DllHandle := 0;
end;
end;
Code: Select all
type TMadExceptFinalize = procedure;
var MadExceptFinalize : TMadExceptFinalize;
5) Run the application, it works as expected
6) Close the application: it hangs.
===> Using the debugger, I see that FreeLibrary never returns.
Removing madExcept from calling application: works.
Removing madExcept from dll: works.
Using madExcept in both DLL and calling application: fails
Any solution or workaround available?
Thanks
Re: FreeLibrary, madExcept and win8
This DLL helper thread shutdown dilemma is one I've seen Raymond Chen of Microsoft blog about a number of times.
https://devblogs.microsoft.com/oldnewthing/?p=19233
https://devblogs.microsoft.com/oldnewthing/?p=99675
https://devblogs.microsoft.com/oldnewthing/?p=2733
https://devblogs.microsoft.com/oldnewthing/?p=19233
https://devblogs.microsoft.com/oldnewthing/?p=99675
https://devblogs.microsoft.com/oldnewthing/?p=2733
Re: FreeLibrary, madExcept and win8
Thanks Zunster. Those articles are very interesting, especialy the last one. But what is exposed is not for me, the application writer, but for Madshi to revise madExcept code. Madshi, are you listening?
Regards,
F. Piette
Embarcadero MVP
Regards,
F. Piette
Embarcadero MVP
Re: FreeLibrary, madExcept and win8
Do you have freeze checking activated in the DLL? If so, please try deactivating it. Does that help?
Re: FreeLibrary, madExcept and win8
No freeze checking activated in the DLL nor in the application.
Re: FreeLibrary, madExcept and win8
Can you reproduce the problem in a "simple" test project?