Correct way to unhook APIs

c++ / delphi package - dll injection and api hooking
Post Reply
leochou
Posts: 3
Joined: Wed Mar 03, 2021 2:31 pm

Correct way to unhook APIs

Post by leochou »

Hi,
I'm using MadCHook driver to inject A.dll into some application. A.dll does nothing but loads B.dll and C.dll dynamically, which actually hook some APIs. As far as I know, I can't use UninjectLibraryW because the driver only knows about A.dll. The only way I can think of is to create a separate thread in each dll and wait for a global event to unhook APIs and unload itself. When uninstalling my app, I do the following things.
1. Call UninjectAllLibrariesW to stop injecting new processes.
2. Signal a global event. Then each dll would unhook all APIs and call FreeLibraryAndExitThread.
3. Stop MadCHook driver and uninstall it.

I'm not sure if it is the correcy way? Thanks!
leochou
Posts: 3
Joined: Wed Mar 03, 2021 2:31 pm

Re: Correct way to unhook APIs

Post by leochou »

BTW, do I need to call UnhookAPI on every function, or just call FinalizeMadCHook to unhook all APIs in one shot? Thanks!
madshi
Site Admin
Posts: 10365
Joined: Sun Mar 21, 2004 5:25 pm

Re: Correct way to unhook APIs

Post by madshi »

It's not really good to have API hooks in dlls which you load and unload manually, because that makes unhooking very tricky. If you call UninjectLibrary(), madCodeHook will do some internal "magic". Basically it will unhook all APIs automatically for you, and it will do that *outside* of DllMain. Only once all API hooks are fully unhooked, FreeLibrary() will be called. This is necessary because doing stuff in DllMain is not recommended, it can result in deadlocks.

Now if you have your own dlls which you load and unload manually via LoadLibrary/FreeLibrary, this "magic" isn't happening, as a result if you just call FinalizeMadCHook() inside of DllMain, you may get deadlocks.

Is this approach (of using 3 separate dlls of which you only inject one via InjectLibrary()) really necessary? If you absolutely cannot switch to a concept where you inject and uninject all your hooking dlls through madCodeHook injection APIs, then I guess your best bet may be to find a way to unhook all the APIs manually in your B.dll and C.dll before you call FreeLibrary. But you should then wait with your FreeLibrary call until the APIs are fully unhooked. I'm not sure how you should ideally be doing that. madCodeHook solves this by creating a remote thread with dynamically created code.

Creating threads in your hook dll isn't really recommended, either, because your hook dlls are supposed to be mostly "invisible" to the process they're injected into, but secondary threads are not invisible. Plus, such threads would need to be cleanly shut down before the dll can safely be unloaded. So it's tricky, once more. Maybe FreeLibraryAndExitThread() will help somehow?
leochou
Posts: 3
Joined: Wed Mar 03, 2021 2:31 pm

Re: Correct way to unhook APIs

Post by leochou »

Hi madshi,
Thanks for your answer. In my situation, A.dll would talk to a RPC server to decide which dlls should be loaded. This can vary from time to time. Regardless of invisibility, is creating a secondary thread almost the same as creating a remote thread by MadCHook? Upon receiving a unload signal, my thread would unhook all APIs and call FreeLibraryAndExitThread(). The only problem is I'm not sure if FinalizeMadCHook() is a synchronous funtion, which means it would unhook all the APIs before it returns. If not, should I call UnhookAPI() on each func instead? Thanks!
madshi
Site Admin
Posts: 10365
Joined: Sun Mar 21, 2004 5:25 pm

Re: Correct way to unhook APIs

Post by madshi »

There's no fundamental difference between a secondary thread and a remote thread. However, differences can be the point in time when the thread is created (only when uninjection is performed, or already when your DLLs are loaded?) and which code it executes (code inside of your DLL, or dynamically created code outside of your DLL).

FinalizeMadCHook() is a synchronous function, so I guess you could call it to unhook all APIs. But please make very sure you only call it once. If you call it twice (e.g. in order to uninject in a secondary thread + in DllMain(DLL_PROCESS_DETACH)), bad things may happen.
Post Reply