App crashes when invoking hooked LodLibraryExW...

c++ / delphi package - dll injection and api hooking
Post Reply
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

We have a COM Addin for 64-bit version of Outlook 2013.
In our plugin we hook the family of LoadLibraryX() functions from kernel32.dll
Our replacement functions examine any attempted dll loads to identify and prevent loading of specific dlls.
All of the above seems to work nomally until...
Outlook attempts to run spellcheck via msspell7.dll
In this dll it imports LoadLibraryExW() and attempts to invoke it.
The invoke attempt causes Outlook to crash.
MS debug of provided time-travel-trace data indicates that the function table slot where they expect LoadLibraryExW to be does not appear to represent executable code - hence the crash.
I have verified that if we disable our hooking of the family of LoadLibrary function, we no longer see the crash - so it would appear that establishing the hook does something such that the import and subsequent invoke attempt of the function via code in the spell check dll no longer works.

Some more detail (ASCD as mentioned below is a dll used by our plugin and this is where the hooking is implemented):

When MSSPELL7 loads, it imports (what it thinks is) the kernel32!LoadLibraryExW; ASCD intercepts this call and instead of providing a pointer to the location of kernel32!LoadLibraryExW, it provides the memory location of 0x00000001’71990000 which is supposed to be its implementation of that function (its replacement function). Unfortunately, it appears that ASCD is wrong about where its replacement function resides – because there’s NOTHING at that memory location:

Outllook is calling into a dynamic
> function table - basically msspell7.dll has imported kernel32.dll's
> LoadLibraryExW function. It does this as it needs to load a lexicon
> (.lex file) using LoadLibraryExW(). disassembly code as
> follows:
>
> ... call qword ptr [msspell7!_imp_LoadLibraryExW...
>
> On dumping the memory addresses representing the dynamic function
> table MS sees valid entries for other functions, including functions
> from kernel32.dll but the slot where MS expects to see LoadLibraryExW
> is empty.

Here’s the surrounding dynamic function table - the slot containing 00000001`71990000 is where LoadLibraryExW replacement is expected:
0:000> dps 000007fe`d313b288-0x10
000007fe`d313b278 00000000`777ec7f0 kernel32!EnumSystemLocalesAStub
000007fe`d313b280 00000000`777b8550 kernel32!IsValidLocaleStub
000007fe`d313b288 00000001`71990000
000007fe`d313b290 00000000`779e38f0 ntdll!RtlDeleteCriticalSection
000007fe`d313b298 00000000`777b6e40 kernel32!InitializeCriticalSectionExStub
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by madshi »

Are you using the latest madCodeHook version (3.1.13 or 4.0.0)?

Are you using any special flags when calling HookAPI()?
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

According to madCHook.h:

// ***************************************************************
// madCHook.h version: 3.0.2 · date: 2012-08-02
// -------------------------------------------------------------

So I guess we're behind the times!

We do this:

DWORD dwHookFlags = IsMcAfeeLoaded() ? MIXTURE_MODE : 0;

I'll have to run a test to see if IsMcAfeeLoaded() is returning true or false. I'll get back to you on that. That function is implemented as:

inline bool IsMcAfeeLoaded()
{
return ::GetModuleHandle("EntAPI.dll") != NULL;
}
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by madshi »

All the files have different version numbers. The best way to check is the version information page in the documentation. See "madCollectionRoot\madBasic\Help".

May I ask why you're using the MIXTURE_MODE when McAfee is loaded?

My first recommendation would be to check the version number and update if you're not on the latest build. Maybe that will already fix the problem. The MIXTURE_MODE flag seems weird to me, though, and I don't think I would really recommend using that approach. So even if updating to the latest build already helps, I'd still like to discuss the MIXTURE_MODE flag with you and maybe find a better solution.
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

We're setting dwHookFlags to 0
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

According to the version history table in the doc, the latest madCodeHook entry is 3.1.6.

Sorry, I don't know the history behind use of MIXTURE_MODE - I'll try and find out.

Will also look into getting latest version.
Last edited by pjthompson on Thu Sep 08, 2016 4:06 pm, edited 1 time in total.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by madshi »

Argh, you keep editing your post... :wink:
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

With regard to MIXTURE_MODE, apparently we had a problem with McAfee 8 Corporate Edition way back in 2004. The conclusion at that time was that McAfee and Office were both trying to hook the same functions. For whatever reason, MIXTURE_MODE fixed the problem.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by madshi »

Ah, ok. FWIW, if I may suggest, I'd comment out that code. The mixture mode isn't really good. If the problem comes back, maybe you can then ask here in the forum and we'll work on finding a better solution/workaround. 12 years is an eternity in the computer world, so chances are good the original problem was already fixed by either a change in McAfee or madCodeHook in the meanwhile.

But it's your choice, of course. Just my 2 cents.
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

OK, thanks for the information. I'll grab the latest version and build with that and run some tests. Will let you know how that works out.
pjthompson
Posts: 12
Joined: Tue Feb 04, 2014 8:10 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by pjthompson »

As an experiment, I rebuilt my application with the FOLLOW_JMP flag. With this flag, the application behaved normally: in the debugger I observed various dlls getting loaded, including the dlls related to MS Spell Check and the .lex files that the MS dll code loads.

Based on the comments related to this flag, it sounds like this is a less than ideal solution and it is unlikely that MS will change their code to accommodate my needs ;)

Any observations on why this flag worked and any alternate strategies would be appreciated.

Thanks.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Re: App crashes when invoking hooked LodLibraryExW...

Post by madshi »

So just rebuilding with the latest build, without changing anything, didn't help?

The FOLLOW_JMP flag is actually a good solution for the specific situation when another hook library has already hooked the API you want to hook. By default currently the MIXTURE_MODE is used in this situation, but I'm already considering using FOLLOW_JMP instead by default of MIXTURE_MODE. Both have their pros and cons, though. My plan is to improve FOLLOW_JMP a bit further before making it the default mode. But I know several madCodeHook users are using FOLLOW_JMP and have good success with it. So if it's working fine for you, don't hesitate using it.
Post Reply