Page 1 of 1

UnhookCode fails (GetLastError()==0)

PostPosted: Sun Feb 07, 2016 5:55 pm
by TCS
Hey,

I use and injection driver to inject DLLs into processes. Once the DLL has been injected I am hooking functions using HookCode().
Once I uninject all the injected DLLs, each DLL during its cleanup also calls UnhookCode().

The problem is that all the UnhookCode()s fails. I am trying to use ::GetLastError() to get some information but it all return 0.

Any idea how to continue from this point?

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Sun Feb 07, 2016 6:51 pm
by madshi
madCodeHook automatically unhooks all your hooks before DllMain(PROCESS_DETACH) is executed, because unhooking stuff in DllMain would be a stability risk. So the reason why UnhookCode() fails is that the hook you're trying to uninstall was already automatically uninstalled before.

If you look at the demos, none of them manually unhook anything.

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 08, 2016 9:43 am
by TCS
Okay, I removed the UnHook() from the cleanup (which from what I understand now didn't do anything), but the injected process still crashes (not immediately, but after "playing" with it).

The faulting module (in the crash dialog) is "testhook.dll_unloaded". testhook.dll is the actual DLL I unloaded so I guess windows let me know that. It also proves that the hooks were not uninstalled, otherwise it wouldn't know about this DLL anymore.

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 08, 2016 9:51 am
by madshi
It could be the API hooks not being uninstalled, or it could be something else.

Did you use any special flags when calling HookCode?

You're using UninjectLibrary, or are you uninjecting the dll yourself somehow?

Are you creating any threads or windows in your hook dll?

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 08, 2016 10:42 am
by TCS
- Did you use any special flags when calling HookCode?
No

You're using UninjectLibrary, or are you uninjecting the dll yourself somehow?
I am injecting using the injection driver.
I am uninjecting using UninjectAllLibrariesW().

Are you creating any threads or windows in your hook dll?
Windows - No.
Threads - Yes.
In the DLLmain of the injected DLL I am creating a thread that starts "everything". I am doing so to release the loader lock.
There are also other threads, but they all die during the cleanup in the DLL_PROCESS_DETACH. I have verified that before posting the question
Also, the functions I am hooking are rare ones (specifically for the test) like OpenServiceW().

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 08, 2016 11:20 am
by madshi
What makes you think that the API hook is at fault for the crash? Is it just a guess, or is there some evidence for that?

Creating threads in a hook dll is rather dangerous. Threads do not usually die at all during PROCESS_DETACH. Do you terminate them? If you don't terminate them, they usually don't end during PROCESS_DETACH.

Anyway, try these one by one:

1) Make your hook callback functions "empty", meaning just call the original function, nothing else.
2) Comment out the HookCode/API calls completely.
3) Remove all threads you created.

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 08, 2016 11:48 am
by TCS
I still might (not sure) have a problem with not all threads stopping.
Please let me check it out.
I'll update.

Thanks and sorry!

Re: UnhookCode fails (GetLastError()==0)

PostPosted: Mon Feb 15, 2016 2:07 pm
by TCS
Sorry took me some time to answer, I wanted to be sure that's the problem.
It seems there is a resource leak in OpenSSL and than was the problem.

Thanks again!