- madExceptViewer doesn't obey to ReportLeaksNow

delphi package - automated exception handling
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

- madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

Hi,

I'm testing memory leaks detection within a DLL.
The DLL contains my UDFs destinated for IB Server 2009. Among these UDFs I have ReportLeaksNow_UDF that consists, in principle, of 2 lines

Code: Select all

TStringList.Create;   //to assure at least one leak
ReportLeaksNow (true);
I run the DLL attached to ibserver.exe process, in XE7 debugger. I have a breakpoint in ReportLeaksNow_UDF, so I can see that at the end of quite long stored procedure (calling numerous UDFs) the 2nd line of the above code is executed, ReportLeaksNow_UDF takes about a second to execute, new process madExceptViewer appears in Task Manager, but no report is displayed on the screen.

Do I miss something of http://help.madshi.net/madExceptUnit.htm#ReportLeaksNow ?

Kind regards,
JK
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

Can you please check if you can reproduce this in a brand new test project? (If possible without any funny special components like IB Server). If so, can you please upload the test project?
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

Thank you for the courageous proposal to dive into a real test case.

Before preparing it, i.e. writing a multi-thread server (simple, of course) providing services which can be programmed for the server via a DLL, I just want to verify whether I understood correctly the madExcept doc concerning its usage in a DLL. To have your opinion was the modest intention of my previous message.

So, I described shortly in what my test (using Interbase server) consists.
The context of the test isn't as "funny" as realistic. Many database application programmers use UDFs, some of them not only for a simple math or string manipulation.
Question 1. Was my description understandable, should I be more specific to permit you imagine the test?

Then, at the end of my message I wanted to ask whether you notice any discordance with the doc I indicated.
Question 2. Do you find in my description any probable misunderstanding?
Question 3. Are there any other sources you recommend me for this weekend reading? ;-)

Let me know, please, your 3 answers. I hope they will assure me that my further investigations aren't rooted in an elementary misunderstanding.

With kind regards,
JK
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

I'm sorry. What I asked for has a lot of sense, but maybe I should have explained it better to justify the time you'd need to spend:

If there's a new madExceptViewer.exe process in the task manager, but nothing visible, that very much points to something going wrong somewhere. Calling ReportLeaksNow() is supposed to work. If something goes wrong, there are a couple of probable causes:

1) Either there's a general bug in madExcept.
2) Or madExcept is usually working fine, but for some reason not on your development PC.
3) Or madExcept is usually working fine, but for some reason not in this one specific project of yours, or in this specific situation.

When asking for a test project I meant that you just try to create a new empty project, and then maybe in a Button1Click handler just create a leak and call ReportLeaksNow(true). If that works that proves that ReportLeaksNow(true) generally works, and also generally works on your development PC. If it works, you could try to enter a Button1Click to your main project (with IB etc) and check if it works in that situation. If it does, then maybe the situation in which you've been calling it is somehow "special". Or if it doesn't work there, we would know that for some reason ReportLeaksNow(true) doesn't work in your real test project at all, while it does work in an empty test project.

If ReportLeaksNow(true) doesn't work for you in a test project, either, you could try ReportLeaksNow(false). Maybe that works? Either in a test project or in your real project. If that doesn't work, either, then ReportLeaksNow() seems to be totally broken for you, for some reason.

All of these tests shouldn't cost you more than a couple of minutes, and should us bring several steps nearer to finding the likely cause of the issue.

I understand that you might have wanted me to do some digging on my own, before I ask you to try a test project. But please understand that at least 90% of the support requests I receive in the end prove to not be madExcept's fault, so because of that I usually ask madExcept users to do some more tests to gather more information. As explained above, those tests wouldn't cost you a lot of time. Probably less time than it took me to write this post... :wink:
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

madshi,

At last I have possibility to return to this thread.
When I compile One-Button project I get the *warning* message "Mapping binary file failed (2)".
I wonder why it's presented as warning and not *error* and where in doc it's explained.
Does there exist a kind of Check List enumerating *all* conditions to satisfy in order to have madExcept able reporting leaks.
For the moment, after the compilation, the project doesn't report anything.

Could you point me, please, in the right direction.
JK
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

I've now created a test project on my own, and it appears to work fine here, with your code, in a DLL. See here:

http://madshi.net/LeakTest.rar

I did notice one problem, though: When I first created the test project, the DLL didn't want to "accept" the leak reporting. I had to reload the project to make the setting stick, for some weird reason. I've not been able to reproduce this problem, though. If your DLL still doesn't want to report leaks, try to open it with the freeware "Resource Hacker" and check the resource "RCData\TMADEXCEPT". Does it say "Enabled = True" and "ReportLeaks = True"?

To answer your question: Leak reporting should work fine if you enable it in a 32bit project. There are no funny requirements, except that the madExcept32.dll needs to be available. As long as madCollection is properly installed, the DLL should be automatically available and found by madExcept. If you want to do leak reporting on other PCs where madCollection is not installed, you need to distribute the madExcept32.dll with your exe, in the same folder as your exe/dll files.
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

madshi,

Thank you very much for your very kind collaborative attitude.

Your LeakTest confirms that madExcept.ReportLeeksNow is effective. But, unfortunately, not always. And very probably it is not your fault. This observation is suggested by further tests I did in my "non-academic" :wink: environment.

Let me recall:

1. MyUdfs.dll is registered with an IB database (say, DB) addressed by MyApp.exe, see 2.. DB is accessed by MyApp via IB Server 2009. When MyApp connects to DB, the IB server becomes aware of UDFs (User Defined Functions) registered with DB, and knows that UDFs are hosted by MyUdfs.dll. The server loads the dll when, for the first time, an SQL code executed by the server calls an UDF exported by MyUdfs.dll.
MyUdfs.dll was compiled (Delphi XE8) to be "madExcept aware" (i.e. during MyUdfs "load_&_init" madExcept starts memory allocations monitoring, at least of allocations done by MyUdfs code).
BTW, madExcept is the very first unit in the "uses" clause of MyUdfs.pas. In the same clause there exist several units, one of them is MyUdfs_U1.pas. MyUdfs_U1, in its "initialization" section allocates an ObjectList and, symmerically, frees it in its "finalization" section. I hope that madExcept monitoring takes into account both allocation and deallocation of the ObjectList. Am I right?

2. MyApp.exe is an application addressing DB (via the IB server); for my tests it was compiled (Delphi XE8) without "madExcept awarness". MyApp implements certain tasks - let us call them "DB-calculations" - that are done via Stored Procedures hosted in DB and executed by the IB server.
The particularity is that the only Stored Procedures calling UDFs hosted in MyUdfs are those related to (i.e. launched by) DB-calculations.


My tests concern only one DB-calculation, hence one stored procedure (SP) to which I added, as the last line, a call to ReportLeaksNow_UDF (more about it below), a new UDF that I included in MyUdfs.dll.
During the tests
- MyUdfs is compiled with debugging information,
- MyUdfs is run as attached, within debugger, to the process ibserver.exe,
- a breakpoint is set within ReportLeaksNow_UDF code,
- MyApp.exe is run without debugger,
- in MyApp.exe I launch manually the DB-calculation,
- after several seconds the debugger (in ibserver.exe process) stops on the breakpoint,
- I press F9.
What happens next depends on the below test variant ...

Test 1
Leaks reporting UDF is declared

Code: Select all

procedure ReportLeaksNow_UDF;
begin
  {$IFDEF madExcept}
  madExcept.ReportLeaksNow;  //breakpoint line
  {$ENDIF}
end;
After leaving ReportLeaksNow_UDF nothing happens. :(
Let me add that now, contrary to what I experienced on May 22, I see no madExceptViewer process in Task Manager (mystery?).

Test 2
Leaks reporting UDF is declared

Code: Select all

procedure ReportLeaksNow_UDF;
begin
  {$IFDEF madExcept}
  madExcept.SetLeakReportFile (a_file_pathName);
  madExcept.ReportLeaksNow;  //breakpoint line
  {$ENDIF}
end;
After leaving ReportLeaksNow_UDF the report appears in the indicated file, ready for dragging to madExceptViewer. :D
My guess: IB prevents UDFs from launching a new process (e.g. madExceptViewer). If it's the case, I congratulate the IB team (no irony!); I am not qualified to do any investigation on so deep level.

Test 3
Leaks reporting UDF is declared

Code: Select all

procedure ReportLeaksNow_UDF;
begin
  {$IFDEF madExcept}
  madExcept.SetLeakReportFile (a_file_pathName);
  {$ENDIF}
end;
and the line

Code: Select all

  {$IFDEF madExcept}
  madExcept.ReportLeaksNow;  //breakpoint line
  {$ENDIF}
is added at the end of the "finalization" section of MyUdfs_U1.pas (i.e. almost at the end of "fini_&_unload" stage of the dll stay within ibserver.exe process.
After passing the first break, when the DB-calculation was finished, I have quit MyApp.exe apparently provoking MyUdfs.dll unloading; debugger stopped in "finalization" section of MyUdfs_U1.pas and F9 resulted - as in Test 2 - in leaks report written to the indicated file. Moreover the report should be of better quality, i.e. less "apparent" leaks. :D

Test 4
Leaks reporting UDF is declared as in Test 3 but nothing is added to the "finalization" section of MyUdfs_U1.pas.
I hope that madExcept unit, seeing that it's going to be unloaded, will call itself a kind of ReportLeaksNow.
But ..., no room for dreams, no file with leak report. :(

That's all about my tests. From the purely pragmatic point of view I can say that I've got a solution (Test 3). For you as the author my description can be useful, at least I hope. Maybe you are able to "correct" madExcept in the spirit of Test 4.

Concerning your answer
To answer your question: Leak reporting should work fine if you enable it in a 32bit project. There are no funny requirements, except that the madExcept32.dll needs to be available. As long as madCollection is properly installed, the DLL should be automatically available and found by madExcept.
it can satisfy only somebody who needs no understanding. For various reasons, my installations, files organization and unwillingness to let anybody to tinker my code, have very few in common with the "default" case. I'm not proud of that but I consider it optimal in my situation. But it needs my understanding where, given a tool as madExcept, the tool expects to find particular file(s) or needs a particular setting. This is just an example of info which you could present in form of the Check List I've mentioned.

Besides the Check List I see just few very easy improvements, so the tool seems very good. But that will be subject of another post.

Kind regards,
JK
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

In theory madExcept should automatically create a leak report when a DLL is unloaded, as long as leak reporting was turned on in the madExcept settings of that DLL. Practically, madExcept hooks DllMain of every DLL which has leak reporting enabled, and when DllMain is called with DLL_PROCESS_DETACH, leak reporting is activated. This is supposed to make sure that everything that still gets freeze during unit finalization is not reported as a leak. At one point I also had tested all this and it worked just fine. Not sure why it doesn't seem to work in your case. Maybe there's something special about your DLL (e.g. maybe it doesn't have an entry point at all, which would be extremely weird, though), or maybe this part of leak reporting got broken at some point, I don't know.

About your check list: Just search in madExcept.pas for "madExcept32.dll" and you'll find the code which tries to locate/load the dll. You'll understand quicker that way than if I write up a long description.
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

Am I supposed to have access to madExcept.PAS?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

If you have a commercial license, then yes. Don't you?
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

madshi wrote:If you have a commercial license, then yes. Don't you?
Not yet, I'm still trying to determine how I should use madExcept in my environment.
Practically, madExcept hooks DllMain of every DLL which has leak reporting enabled, and when DllMain is called with DLL_PROCESS_DETACH, leak reporting is activated.
I wonder whether MyUdfs.dll is reporting enabled? How I can check it?
AFAIU, Delphi compiler deals with DllMain and DLL entry point. Should I write DllMain myself and how to do that to match madExcept expectations?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

Here's the code that loads/locates madExcept32.dll:

Code: Select all

    dll := LoadLibrary('madExcept32.dll');
    if dll = 0 then
      dll := LoadLibraryW(PWideChar(ExtractFilePathW(ModuleName(HInstance)) + 'madExcept32.dll'));
    if dll = 0 then begin
      s1 := RegReadStr(HKEY_LOCAL_MACHINE, 'Software\madshi\madCollection');
      if s1 = '' then
        s1 := RegReadStr(HKEY_CURRENT_USER, 'Software\madshi\madCollection');
      dll := LoadLibraryW(PWideChar(s1 + '\madExcept\Dlls\madExcept32.dll'));
    end;
Yes, DllMain should be provided by Delphi, usually. There should be nothing you need to do yourself.

You can use the freeware "Resource Hacker" to look at the "RCData\TMADEXCEPT" resource in your DLL. It contains the madExcept settings, as I already explained earlier.
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

Resource Hacker shows in MyUdfs.dll

Code: Select all

object madExcept: TMadExcept
  Enabled = True
  NoSettings = False
  CheckFileCrc = True
  CheckFreeze = False
  FreezeTimeout = 60000
  ReportLeaks = True
  WindowsLogo = False
  CrashOnBuffer = False
  CrashOnUnderrun = False
  AutoSave = True
  AutoSaveIfNotSent = True
  AutoSend = False
  AutoSendBox = False
  AutoClip = False
  PauseThreads = False
  PlWaitBox = True
  AutoContinue = False
  AutoRestart = 0
  AutoClose = 0
  SendBtnVis = True
  SaveBtnVis = False
  PrintBtnVis = False
  ShowBtnVis = True
  ContinueBtnVis = True
  RestartBtnVis = True
  CloseBtnVis = True
  FocusedBtn = 0
  SendAssis = 'SendAssistant'
  SaveAssis = 'SaveAssistant'
  PrintAssis = 'PrintAssistant'
  AutoShowBugRep = False
  UglyBtns = False
  SendInBackgr = True
  UploadViaHttp = False
  HttpServer = ''
  HttpSsl = False
  HttpPort = 80
  HttpAccount = ''
  HttpPassword = ''
  FogBugz = False
  BugZilla = False
  Mantis = False
  BugTrAccount = ''
  BugTrPassword = ''
  BugTrProject = ''
  BugTrArea = ''
  BugTrAssignTo = ''
  MailAsSmtpServer = False
  MailAsSmtpClient = False
  SmtpServer = ''
  SmtpSsl = False
  SmtpTls = False
  SmtpPort = 25
  SmtpPort = 0
  SmtpAccount = ''
  SmtpPassword = ''
  MailViaMapi = True
  MailViaMailto = True
  MailAddr = ''
  AttachBugRep = True
  AttachBugRepFile = True
  DelBugRepFile = True
  BugRepSendAs = 'bugreport.txt'
  BugRepZip = ''
  ScrShotDepth = 8
  ScrShotAppOnly = False
  ScrShotSendAs = 'screenshot.png'
  ScrShotZip = ''
  AddAttachs = ''
  BugRepFile = 'bugreport.txt'
  AppendBugReps = True
  BugRepFileSize = 100000
  NoDupExcepts = True
  NoDupFreezes = True
  DupExceptDef = 1
  DupFreezeDef = 2
  ListThreads = True
  CpuRegs = True
  StackDump = True
  ShowDisAsm = True
  HideUglyItems = False
  ShowRelAddrs = True
  ShowRelLines = True
  FormatDisAsm = False
  LimitDisAsm = 5
  Plugins = 'modules|processes|hardware'
  F1Classes = 'EDBEditError'
  F1NoBugRep = True
  F1NoScrShot = True
  F1NoHandlers = True
  F1NoSuspend = True
  F1ShowCfg = 3
  F1Assis = ''
  F2Classes = ''
  F2NoBugRep = False
  F2NoScrShot = False
  F2NoHandlers = False
  F2NoSuspend = False
  F2ShowCfg = 0
  F2Assis = ''
  GnNoBugRep = False
  GnNoScrShot = False
  GnNoHandlers = False
  GnNoSuspend = False
  GnShowCfg = 0
  GnAssis = ''
  Assistant1 = 'SendAssistant|Send Assistant|ContactForm|DetailsForm|ScrShotForm'
  Assistant2 = 'SaveAssistant|Save Assistant|ContactForm|DetailsForm'
  Assistant3 = 'PrintAssistant|Print Assistant|ContactForm|DetailsForm'
  TitleBar = '%appname%'
  ExceptMsg = 'An error occurred in the application.'
  FrozenMsg = 'The application seems to be frozen.'
  BitFaultMsg = 'The file "%modname%" seems to be corrupt!'
  SendBtnTxt = 'send bug report'
  SaveBtnTxt = 'save bug report'
  PrintBtnTxt = 'print bug report'
  ShowBtnTxt = 'show bug report'
  ContinueBtnTxt = 'continue application'
  RestartBtnTxt = 'restart application'
  CloseBtnTxt = 'close application'
  OkBtnTxt = '&OK'
  DetailsBtnTxt = '&Details'
  PlWaitTitle = 'Information'
  PlWaitText = 'Please wait a moment...'
  BugTrTitle = '%appname%, %exceptMsg%'
  BugTrDescr = 'error details: '#13#10'%errorDetails%'
  MailSubj = 'bug report'
  MailBody = 'please find the bug report attached'
  SendBoxTitle = 'Sending bug report...'
  PrepAttMsg = 'Preparing attachments...'
  MxLookMsg = 'Searching for mail server...'
  ConnMsg = 'Connecting to server...'
  SendMailMsg = 'Sending mail...'
  FieldMsg = 'Setting fields...'
  SendAttMsg = 'Sending attachments...'
  SendFinalMsg = 'Finalizing...'
  SendFailMsg = 'Sorry, sending the bug report didn'#39't work.'
end
Does it contains any hint why, in Test 4, dll unloading doesn't provoke leaks report writing to the indicated file.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by madshi »

No, looks fine.
jkomorowski
Posts: 14
Joined: Sun Dec 28, 2014 12:30 pm

Re: - madExceptViewer doesn't obey to ReportLeaksNow

Post by jkomorowski »

Since yesterday my Test 3 approach seemed a "solution", I did two modifications to check whether it's reliable.
1. In one of UDFs I injected leak line

Code: Select all

TStringList.Create;
and I have set breakpoint on it.
2. The parameter "true" in the Test 3 call of ReportLeaksNow I changed to "false" to have leak report as rich as possible.
Moreover I have set breakpoint on that call.

Then I followed the Test 3 scenario. Debugger stopped 6 times on the leaking line and, after quitting MyApp.exe, it stopped on the ReportLeaksNow call. All 7 breaks were in the same thread thread $5384.

The resulting leak report
SP_005107_074.zip
(29.99 KiB) Downloaded 292 times
is quite big but mentions only one TString leaked and in thread $1508.
In a sense, this report ignores the forced leaks.

What do you think about that?
Has madExcept a kind of "private log" which could cast some light on the phenomenon?
Post Reply