Randon Access Violation on dhcpcsvc6.DLL

delphi package - automated exception handling

Randon Access Violation on dhcpcsvc6.DLL

Postby detinho » Thu Aug 08, 2013 1:13 pm

Hi.
We're having some random access violation when runinng our application on Windows 2008 Server(fully updated). According to my understending of the madExcept logs, the errors are occurring in the madExcept's own code.
Here are three logs of the errors: https://dl.dropboxusercontent.com/u/221 ... ordhcp.zip.

All three logs have this call stack, with some small variations:

Code: Select all
date/time         : 2013-08-08, 08:49:44, 130ms
computer name     : SERVER201
wts client name   : DARKA-F2427F8A7
user name         : cd07
registered owner  : Usuário do Windows
operating system  : Windows 2008 R2 x64 Service Pack 1 build 7601
system language   : Portuguese
system up time    : 13 hours 54 minutes
program up time   : 1 minute 20 seconds
processors        : 16x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
physical memory   : 12771/32755 MB (free/total)
free disk space   : (C:) 369,49 GB
display mode      : 1024x768, 16 bit
process id        : $6524
allocated memory  : 84,20 MB
executable        : autcom.exe
exec. date/time   : 2013-08-07 15:41
version           : 27.18.365.85
compiled with     : Delphi 2006/07
madExcept version : 4.0.7
callstack crc     : $776e705e, $08562d73, $08562d73
exception number  : 2
exception class   : EAccessViolation
exception message : Access violation at address 73601C89 in module 'dhcpcsvc.DLL'. Write of address 73601C89.

thread $6a0c:
73601c89 +15c dhcpcsvc.DLL           DhcpIsEnabled
741c6a84 +037 IPHLPAPI.DLL           GetAdaptersAddresses
00462ac5 +00d autcom.exe   madExcept CallThreadProcSafe
0046321b +037 autcom.exe   madExcept UserWorkItemExceptFrame
76d633a8 +010 kernel32.dll           BaseThreadInitThunk

Main ($6544):
00000000 +ffbd19c4 autcom.exe madStackTrace +0 StackAddrToStr
>> stack will be calculated soon


Does this have a quick workaround (either with some madExcept configuration or altering my code) or is some bug/incompatiblity?
detinho
 
Posts: 2
Joined: Thu Aug 08, 2013 12:56 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Thu Aug 08, 2013 2:30 pm

It's weird that you don't get a proper callstack for the main thread. That could be a bug in madExcept, but this is probably of no consequence for this specific problem.

The crash occurs when a thread is calling "IPHLPAPI.GetAdaptersAddresses()". The crash is occuring somewhere inside of the IPHelper dll. This is a system dll. Does your code call "GetAdaptersAddresses()" somewhere? Or maybe some of the third party components you're using? madExcept itself does not call this function.

The "madExcept.CallThreadProcSsafe" and "madExcept.UserWorkItemExceptFrame" items in the callstack have to be there. That's how madExcept is able to automatically catch exceptions in secondary threads for you. These callstack items do not in any way indicate that madExcept is causing the crashes.
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby detinho » Thu Aug 08, 2013 6:56 pm

I searched through all the project and it's components and none of them uses the GetAdaptersAddresses.
The fact that there is no call stack for the main thread, is that we unchecked the "call stack of all running threads" option.

To see a full log, please dowload again from https://dl.dropboxusercontent.com/u/221 ... ordhcp.zip that now has some logs from yesterday.

Thanks!
detinho
 
Posts: 2
Joined: Thu Aug 08, 2013 12:56 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Thu Aug 08, 2013 7:15 pm

Well, I can't say from the bug report who's calling GetAdaptersAddresses and why. But somebody does, inside of your process. It could be an indirect call. Meaning some of your code (or 3rd party code, or some Delphi RTL/VCL unit) calls some other win32 API and that internally calls GetAdaptersAddresses. Maybe it's somehow related to that Soap Http stuff? GetAdaptersAddresses has something to do with network, IP etc... I don't know, just guessing around here...

I wish I could tell you more, but there isn't really any more information in the crash report for this specific problem... :(
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby Chris08 » Wed Nov 20, 2013 3:06 pm

Hi,

we got the same AV but with the MainThread-Callstack. Seems that the application is shutting down. Maybe this helps to track down the problem.
We assume that the problem has something todo with W2008 Server, maybe one should set the tsaware-flag?

http://stackoverflow.com/questions/1437 ... minal-serv


Code: Select all
date/time          : 2013-10-28, 09:40:24, 696ms
computer name      : <>
wts client name    : <>
user name          : <>
registered owner   : Windows-Benutzer
operating system   : Windows 2008 R2 x64 Service Pack 1 build 7601
system language    : German
system up time     : 20 days 21 hours
program up time    : 3 seconds
processors         : 4x Intel(R) Xeon(R) CPU E5405 @ 2.00GHz
physical memory    : 1653/4095 MB (free/total)
free disk space    : (C:) 95,34 GB (I:) 113,11 GB
display mode       : 1424x923, 16 bit
process id         : $2ef8
allocated memory   : 27,46 MB
largest free block : 1,35 GB
command line       : "I:\<Path>\webupgrade.exe" UPDATEAVAILABLE C:\Users\adminsd\AppData\Local\<Path>\DBT73ED.LOG
executable         : webupgrade.exe
exec. date/time    : 2013-10-22 12:09
version            : 0.9.0.123
compiled with      : Delphi 2010
madExcept version  : 4.0.8.1
callstack crc      : $776e705e, $63a771a1, $af67e584
exception number   : 1
exception class    : EAccessViolation
exception message  : Access violation at address 740F1C89 in module 'dhcpcsvc.DLL'. Write of address 740F1C89.

thread $3698:
740f1c89 +15c dhcpcsvc.DLL             DhcpIsEnabled
74126a84 +037 IPHLPAPI.DLL             GetAdaptersAddresses
0047afa5 +00d webupgrade.exe madExcept CallThreadProcSafe
0047b6ff +037 webupgrade.exe madExcept UserWorkItemExceptFrame
74bc3368 +010 kernel32.dll             BaseThreadInitThunk

main thread ($2904):
76f6f8ca +0e ntdll.dll                      NtWaitForSingleObject
752f1497 +92 KERNELBASE.dll                 WaitForSingleObjectEx
74bc118f +3e kernel32.dll                   WaitForSingleObjectEx
74bc1143 +0d kernel32.dll                   WaitForSingleObject
00471269 +69 webupgrade.exe madExcept       CloseHandleExceptionThread
0047f1af +53 webupgrade.exe madExcept       Close
0047f33a +2e webupgrade.exe madExcept       Finalization
00407056 +3e webupgrade.exe System    12622 FinalizeUnits
0047a368 +54 webupgrade.exe madExcept       InterceptFinalizeUnits
0047a377 +07 webupgrade.exe madExcept       InterceptHalt0FinalizeUnits
74bc3368 +10 kernel32.dll                   BaseThreadInitThunk

thread $b9c:
76f71f3f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $a04:
76f70156 +0e ntdll.dll     NtWaitForMultipleObjects
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $30f8:
76f71f3f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $33ec: <priority:2>
74e57c18 +45 USER32.dll               GetMessageA
0047afa5 +0d webupgrade.exe madExcept CallThreadProcSafe
0047b00f +37 webupgrade.exe madExcept ThreadExceptFrame
74bc3368 +10 kernel32.dll             BaseThreadInitThunk
>> created by main thread ($2904) at:
745a6c8b +00 winmm.dll

thread $eb4:
76f71f3f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $2684:
76f71f3f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $12dc:
76f6f8ca +0e ntdll.dll                NtWaitForSingleObject
752f1497 +92 KERNELBASE.dll           WaitForSingleObjectEx
74bc118f +3e kernel32.dll             WaitForSingleObjectEx
74bc1143 +0d kernel32.dll             WaitForSingleObject
0047afa5 +0d webupgrade.exe madExcept CallThreadProcSafe
0047b00f +37 webupgrade.exe madExcept ThreadExceptFrame
74bc3368 +10 kernel32.dll             BaseThreadInitThunk
>> created by main thread ($2904) at:
75df1102 +00 WININET.dll

thread $2ba8:
76f70156 +00e ntdll.dll                NtWaitForMultipleObjects
752f15e3 +0fa KERNELBASE.dll           WaitForMultipleObjectsEx
74bc19f7 +089 kernel32.dll             WaitForMultipleObjectsEx
75cb4d21 +065 WS2_32.dll               WSALookupServiceNextW
75cb4a94 +20b WS2_32.dll               GetAddrInfoW
0047afa5 +00d webupgrade.exe madExcept CallThreadProcSafe
0047b6ff +037 webupgrade.exe madExcept UserWorkItemExceptFrame
74bc3368 +010 kernel32.dll             BaseThreadInitThunk

thread $2a48:
76f71f3f +0b ntdll.dll     NtWaitForWorkViaWorkerFactory
74bc3368 +10 kernel32.dll  BaseThreadInitThunk

thread $494:
76f6fd8a +0e ntdll.dll                NtDelayExecution
752f3bc2 +5f KERNELBASE.dll           SleepEx
752f4493 +0a KERNELBASE.dll           Sleep
0047afa5 +0d webupgrade.exe madExcept CallThreadProcSafe
0047b00f +37 webupgrade.exe madExcept ThreadExceptFrame
74bc3368 +10 kernel32.dll             BaseThreadInitThunk
>> created by thread $30f8 at:
74d0da8e +00 ole32.dll

thread $3634: <priority:1>
76f6f952 +0e ntdll.dll                NtRemoveIoCompletion
0047afa5 +0d webupgrade.exe madExcept CallThreadProcSafe
0047b00f +37 webupgrade.exe madExcept ThreadExceptFrame
74bc3368 +10 kernel32.dll             BaseThreadInitThunk
>> created by main thread ($2904) at:
731aa33f +00 mswsock.dll

disassembling:
[...]
740f1c62   mov     [ebp-$270], esi
740f1c68   mov     [ebp-$26c], esi
740f1c6e   mov     [ebp-$268], ecx
740f1c74   mov     dword ptr [ebp-$264], $238
740f1c7e   mov     [ebp-$284], esi
740f1c84   call    -$172 ($740f1b17)      ; NsiGetAllParametersEx (NSI.dll)
740f1c84
740f1c89 > mov     edi, eax
740f1c8b   cmp     edi, esi
740f1c8d   jnz     loc_740f1ccf
740f1c8d
740f1c8f   lea     eax, [ebp-$250]
740f1c95   push    eax
740f1c96   lea     eax, [ebp-$24]
740f1c99   push    eax
[...]


Thanks for your help!
Last edited by Chris08 on Wed Nov 20, 2013 5:06 pm, edited 1 time in total.
Chris08
 
Posts: 26
Joined: Wed Mar 01, 2006 3:13 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Wed Nov 20, 2013 3:22 pm

Looks like the dhcpcsvc6.DLL might already be unloaded in the moment when "IpHlpApi.GetAdaptersAdresses()" internally tries to call "hdcpcsvc6.DhcpIsEnabled()". Or maybe not. Can't say because the bug report is incomplete (no module list). It's also possible that the dll is still loaded, but in the process of being unloaded or something.
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby Chris08 » Wed Nov 20, 2013 5:07 pm

Yes, unfortunatly, no module list.
I just added the disasm-part.
Chris08
 
Posts: 26
Joined: Wed Mar 01, 2006 3:13 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Wed Nov 20, 2013 5:19 pm

That is extremely weird. Taking the disasm into account, the bugreport suggests that the 0x740f1xxx code page of dhcpcsvc6.dll was changed to "non executable" while a thread was still running through that code. Not sure how this can happen. I would say the most likely situation is that some other thread is trying to unload that dll while the crashing thread is still executing code in the dll. Or maybe some other thread has changed the page access rights of the dll for some reason (could be a bug in the code or something).
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby rowwt » Fri Jan 17, 2014 12:50 pm

Hi,
I get the same error. It is happening on a citrix terminal server environment. We are using MapiSendMail to open the default mail client. The customer has outlook. Outlook popups having all the data we provide in the code. But randomly, the application is crashing behind the outlook's new mail window. Madexcept is also failing to send us the report so i have paid a visit to customer and did some screen shots with the error and with the call stack. I've attached the files.
Do you have any clue on why is happening? We get this error only on citrix and we are not able to reproduce it on our test/devel computers.
Attachments
madexcept_error.png
madexcept_error.png (32.45 KiB) Viewed 7409 times
madexcept_stack.png
madexcept_stack.png (50.21 KiB) Viewed 7409 times
rowwt
 
Posts: 2
Joined: Fri Jan 17, 2014 11:32 am

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Fri Jan 17, 2014 1:12 pm

According to the bug report the crash occurs in a thread created by "mso.dll" which is a part of Microsoft Office. I suppose this could be the dll handling the MAPI transport from your process to MS Outlook. This is just a guess on my part, though. If I'm guessing correctly, the bug could be in "mso.dll". It might make sense to switch to a better mail sending method. E.g. SMTP or HTTP upload. You could also try updating MS Office with the latest service packs in the hope that this might fix the problem.
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby zunzster » Fri Jan 17, 2014 9:32 pm

We see this issue quite regularly as we use MAPI and lots of our users run our application on TS.

Office (and Outlook in particular) is not DEP safe on Windows 2008 R2. http://support.microsoft.com/kb/2028367

If you're wondering why this doesn't occur on non-server versions of Windows (e.g. XP, Vista, Win7, etc.),
it's because *only* 'system' processes run with DEP enabled by default in desktop versions, whereas DEP is enabled
on *all* processes by default in server versions.

TS (and Citrix) are the classic case where a server version of Windows ends up hosting desktop applications and thus
where you see this annoying issue.

Now in the above KB, Microsoft recommends exempting Outlook.exe from DEP to workaround the issue.
However, when talking via MAPI to Outlook, the problematic non DEP-safe code will be running in *your* address space,
thus you will need to exempt *your* processes exe from DEP checking or suffer these spurious AVs.
zunzster
 
Posts: 46
Joined: Wed Oct 29, 2008 3:43 am

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby madshi » Fri Jan 17, 2014 11:53 pm

Thanks for chiming in, that's good to know!
madshi
Site Admin
 
Posts: 9203
Joined: Sun Mar 21, 2004 5:25 pm

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby rowwt » Mon Jan 20, 2014 10:31 am

Very good news indeed. I hope this will solve our issue. Thank you.

While searching on the internet about the issues on terminal server environments, i also found that is possible to make your application TS-aware by adding this flag: {$SetPEOptFlags IMAGE_DLLCHARACTERISTICS_TERMINAL_SERVER_AWARE}
I had no time to test this. Do you know anything about it? Will DEP consider the application safe if it was built with this flag?
rowwt
 
Posts: 2
Joined: Fri Jan 17, 2014 11:32 am

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby zunzster » Mon Jan 20, 2014 8:08 pm

My understanding is that flag is about *your* application and whether it needs the TS app compatibility DLL loaded. http://msdn.microsoft.com/en-us/library ... 85%29.aspx
As far as I know, it has no effect on DEP or on other peoples code, in this case, Outlook's DLLs.
zunzster
 
Posts: 46
Joined: Wed Oct 29, 2008 3:43 am

Re: Randon Access Violation on dhcpcsvc6.DLL

Postby Lane » Sat Sep 10, 2016 6:59 am

rowwt wrote:Very good news indeed. I hope this will solve our issue with male extra now. Thank you.

While searching on the internet about the issues on terminal server environments, i also found that is possible to make your application TS-aware by adding this flag: {$SetPEOptFlags IMAGE_DLLCHARACTERISTICS_TERMINAL_SERVER_AWARE}
I had no time to test this. Do you know anything about it? Will DEP consider the application safe if it was built with this flag?


Where abouts do you place this flag in the code?
Lane
 
Posts: 1
Joined: Fri Sep 09, 2016 6:10 am

Next

Return to madExcept

Who is online

Users browsing this forum: No registered users and 1 guest