Hooking 32 bit apps in Windows XP 64-bit

[Cliff]

Hi.

Some of the users of a program I've written have complained that the hooking mechanisms don't function when the program is run in the 64-bit version of Windows. Both the target program and the hook (of course) is in 32 bit, so they both are run under the WOW64 simulated 32 bit enviornment that the 64 bit version of Windows provides.

I'm just wondering, do you know if madcodehook just doesn't work in the 64 bit version of Windows? And if so, is it possible to make it work, or is there some issue with hooking in general under WOW64?

Thanks
-Cliff

[madshi]

Ouch.

madCodeHook doesn't support 64bit OSs yet, unfortunately. It's a whole new world (at system level), so it's quite a big project for me. I'll be working on 64bit support very soon, but it will take some time.

Sorry.

[madshi]

P.S: If you're satisfied with only hooking 32bit processes, you could try to (mis)use SetWindowsHookEx to inject your hook dll into the running processes. madCodeHook's HookAPI/HookCode functions should work just fine. It's just the DLL injection which is likely to fail.

[Cliff]

I'll try using SetWindowsHookEx...thanks ^^. It's not a huge problem yet, I doubt 64-bit Windows will be mainstream for a few years.

One other question. A few thousand people use my program...and for the vast mojority it works flawlessly, but for a few the hooking only partially works or not at all. Sometimes it can be attributed to bizarre situations like the DLL to be injected being missing, but for others I've gone through some pretty heavy debugging, and a couple even have virtually identical system setups as my own, including same version/service pack of Windows, same graphics card and CPU core, etc.... but the hook just fails. (Both with CreateProcessEx, and InjectLibrary). My only guess has been that there's some driver they have that does hooking on its own which interferes with madcodehook...like possibly a graphics driver that adds a couple buttons to the top window bar for dual monitor features, or something.

So my question is, is there any known conflicts with any particular drivers, software, whatever? What I'm hooking are window creation and manipulation functions, as well as a few DirectX instance creation functions.

Thanks
-Cliff

[madshi]

I don't think there are any problems with drivers, since they work at kernel land, while madCodeHook is strictly user land. However, if another user land hook is active, there might be conflicts. E.g. "Fraps" (do you know that?) hooks DirectX, too, I believe. So in such situation there might be complications. I'm not aware of any specific program or software that makes problems, though.

What exactly fails on those problem PCs? Is it the DLL injection? Or the hooking? If it's the hooking - what does HookAPI return? Does it report that hooking didn't work? Or does it report success, but no hooks are actually getting fired?

[Cliff]

>I don't think there are any problems with drivers, since they work at kernel land, while madCodeHook is strictly user land. However, if another user land hook is active, there might be conflicts. E.g. "Fraps" (do you know that?) hooks DirectX, too, I believe. So in such situation there might be complications. I'm not aware of any specific program or software that makes problems, though.

Yeah, I know fraps...I'll find out if they're using any programs like that.

>What exactly fails on those problem PCs? Is it the DLL injection? Or the hooking? If it's the hooking - what does HookAPI return? Does it report that hooking didn't work? Or does it report success, but no hooks are actually getting fired?

It's definitely the injection, both CreateProcessEx and InjectLibrary return failure. Maybe the SetWindowsHookEx method would work for these people?

Edit: Oh, and CreateProcessEx Does create the process, just doesn't inject successfully.

-Cliff

[madshi]

Oh, if injection is the problem then it shouldn't have anything to do with other programs installing hooks, too.

Are you injecting into one specific process or system/user/session wide. In the latter case: Are you aware that InjectLibrary(specialFlag) needs admin rights?

[Cliff]

>Are you injecting into one specific process or system/user/session wide. In the latter case: Are you aware that InjectLibrary(specialFlag) needs admin rights?

It's one process. My normal injection method is into one process with CreateProcessEx. When that fails, the process still opens, so recently I added a little debug step to try InjectLibrary. Neither work for the people it doesn't work for. One thing I haven't asked about is whether they have admin rights on their computers...I'll ask them. Does CreateProcessEx require admin rights as well?

-Cliff

[madshi]

CreateProcessEx and InjectLibrary(oneSpecificProcessHandle) normally doesn't need admin rights.

Are you using C++ or Delphi? If you're using C++, are you using the dynamic or static lib?

What does InjectLibrary return, if injection fails? TRUE or FALSE? What does GetLastError say?