Size of hook dll is 150 KB - shrink it?

[dcsoft]

In DllMain there could be lots of stuff which was already being executed before the HookAPI call finds out that the callstack of a thread is not ok. Simply unloading the dll is not really a good idea. What if all the other API hooks were already installed and are working just fine? And who guarantees that reinjection a few milliseconds later will solve the callstack problem? It might even be that the hook dll is written in a way that doesn't allow uninjection without crashing. I do have such a dll written myself. There's no problem with it, since I don't ever uninject it myself.

Well, it might not work for all cases, but it would for my hook dll. Perhaps you could add an API called IsSafeToHook(dllname, functionname) which checks the call stacks of running threads and if the API is found return FALSE. Then the caller (me) could decide whether to uninject the dll and try again later. Just an idea, it may not work, but providing guaranteed successful hooking, at least under some limited circumstances, could potentially be a valuable addition to madCodeHook. At any rate, please take it only as a suggestion.

Cheers and Merry Christmas.

-- David

[dcsoft]

Perhaps you could add an API called IsSafeToHook(dllname, functionname) which checks the call stacks of running threads and if the API is found return FALSE. Then the caller (me) could decide whether to uninject the dll and try again later.

Hi Madshi, were you planning to implement this soon? If not, I could do it; I found the StackWalk() API which seems useful for this.

Thanks,
David

[madshi]

I've no plans for implementing that right now.

You don't need to check the whole stack, btw. Should be enough to check each thread's instruction pointer (EIP).

[dcsoft]

I've no plans for implementing that right now.

You don't need to check the whole stack, btw. Should be enough to check each thread's instruction pointer (EIP).

OK, thanks for the tip. I'll let you know how it goes.

-- David

[madshi]

Do you even have a situation where instability can be reproduced? I've not heard of such problems yet. I can reproduce it with an artificial stress test only, not in real life. So it might be hard for you to test, whether your solution improves the situation or not.

[dcsoft]

Do you even have a situation where instability can be reproduced? I've not heard of such problems yet. I can reproduce it with an artificial stress test only, not in real life. So it might be hard for you to test, whether your solution improves the situation or not.

The QA dept. has reported inconsistent problems with both injection and hooking (injection fails sometimes in AOL, hooking seems to fail sometimes in IE and Netscape), but to tell the truth, I haven't determined the causes. It could just be pilot error. Anyway, since this timing problem is known, I thought to start there.

[madshi]

What does "fail" mean? Did your QA get crashes? Or did the injection/hooks simply not work (as of "nothing happened")?

I have several big customers who did extensive QA testing, too, and they didn't find any stability problems. It might depend on which APIs you're hooking, though, because the more often a hooked API is called, the bigger the danger of problems during hook (un)installation.

Btw, the danger is only there for processes which are already running. Newly created processes (which are created after the injection was already started) should not be in any danger, because for such newly created processes the dll is loaded before any secondary thread can even start running.

[dcsoft]

What does "fail" mean? Did your QA get crashes? Or did the injection/hooks simply not work (as of "nothing happened")?

I have several big customers who did extensive QA testing, too, and they didn't find any stability problems. It might depend on which APIs you're hooking, though, because the more often a hooked API is called, the bigger the danger of problems during hook (un)installation.

Btw, the danger is only there for processes which are already running. Newly created processes (which are created after the injection was already started) should not be in any danger, because for such newly created processes the dll is loaded before any secondary thread can even start running.

Let me sit down with the QA this week and see what they're complaining about exactly. The only specific thing I know right now is that he didn't see my hook dll in WAOL.EXE, using Sysinternals' Process Explorer. That means injection failed. He also says that in IE and Netscape it seemed the hook wasn't doing the job, but that could be due to other parts of the system. I'll let you know in more detail what I find.

I'm not installing a system-wide hook. I'm injecting into specific processes once they're started, using a SetWinEventHook() callback to get notified when their windows are created. Perhaps in WAOL.EXE, the first thing it does is send stuff out the Internet (I'm hooking send()), and that is happening when my DLL is injected and HookAPI() is running, I don't know.

Perhaps a system-wide hook is preferrable. But I still need to hook existing processes running before my injector exe starts.

Thanks,
David

[dcsoft]

Ah ya! Your comment in the other thread about madCodeHook initialization made me realize I was not calling InitializeMadCHook() and FinalizeMadCHook()! And I'm using the static lib. My bad.

Maybe this explains the flakiness. I'll let you know.

Thanks,
David

[madshi]

That could explain some problems.