[SOLVED] Hooked function crashes in 32 bit windows
[SOLVED] Hooked function crashes in 32 bit windows
I have a Print Providor Hook - hooks various functions in the local print providor (localspl.dll), it gets the functions by trapping "InitializePrintProvidor" and calling HookCode on the function pointers passed to that.
Works perfectly in Windows 7 64 bit. Dies on its ass in Windows 7 32 bit. As far as I can tell from log statements, it happens immediately after a return from any of my hooked functions.
I've tried minimizing the foot print, reducing it to just one hooked function that is merely a stub - it returns immediately, calling the original function. Still crashes the print spooler.
If I only hook the InitializePrintProvidor, it works fine, apart from doing nothing useful
Feels like a mismatch in calling conventions or somesuch, differences in 32/64 code. I've examined all the declerations and compared with the originals in the windows header, they all match.
One thing - none of them are declared with WINAPI (__stdcall).
Any suggestions as to where to go next?
thanks,
Lindsay
Works perfectly in Windows 7 64 bit. Dies on its ass in Windows 7 32 bit. As far as I can tell from log statements, it happens immediately after a return from any of my hooked functions.
I've tried minimizing the foot print, reducing it to just one hooked function that is merely a stub - it returns immediately, calling the original function. Still crashes the print spooler.
If I only hook the InitializePrintProvidor, it works fine, apart from doing nothing useful
Feels like a mismatch in calling conventions or somesuch, differences in 32/64 code. I've examined all the declerations and compared with the originals in the windows header, they all match.
One thing - none of them are declared with WINAPI (__stdcall).
Any suggestions as to where to go next?
thanks,
Lindsay
Last edited by blackpaw on Wed May 07, 2014 9:56 am, edited 1 time in total.
Re: Hooked function crashes in 32 bit windows
In x64 there is only one calling convention. So whatever calling convention specifier you use doesn't matter. In x86 there really are all kinds of different calling conventions. So my best guess is that the calling convention is wrong. Simply try different ones. If everything goes to working correctly, then you know you found the correct one. Try WINAPI first, it's the most likely one...
Re: Hooked function crashes in 32 bit windows
Thanks madshi, but the actual prototypes declared in windows are just plain c.
I'll take a look at sample sdk code tomorrow and compare.
I'll take a look at sample sdk code tomorrow and compare.
Re: Hooked function crashes in 32 bit windows
Don't hang yourself up on prototype declarations. C++ is so complicated with its defines and includes and headers and project options etc that it's hard to predict which calling convention is really used. Just use trial and error. It will cost you 5 minutes of your time and probably fix the issue.
Re: Hooked function crashes in 32 bit windows
Good point I'll give it a spin
Re: Hooked function crashes in 32 bit windows
It was calling conventions in the end (WINAPI), though I didn't have to edit my code.
For a start off I couldn't, I was hooking function pointers in the PRINTPROVIDOR structure and they had no calling conventions specified, and defaulted to cdecl.
What I need to do was to set __stdcall as the default in the project properties, so the windows functions pointers were stdcall as well. All worked after that.
Thanks - Lindsay
For a start off I couldn't, I was hooking function pointers in the PRINTPROVIDOR structure and they had no calling conventions specified, and defaulted to cdecl.
What I need to do was to set __stdcall as the default in the project properties, so the windows functions pointers were stdcall as well. All worked after that.
Thanks - Lindsay
Re: [SOLVED] Hooked function crashes in 32 bit windows
Windows almost always uses WINAPI. It's very rare that cdecl is used. It's kinda strange, though, that when using the "official" Microsoft header, you're getting problems. Anyway, glad to hear you got it solved. If you want my opinion: I'd rather define the structure yourself, so you don't rely on the project options. Otherwise maybe at some point you recreate the project or something, and suddenly all hell breaks lose because the project reverts to cdecl. Just my 2 cents, of course...