Size of hook dll is 150 KB - shrink it?

[dcsoft]

Tested it myself in BCB5. With packages its 180 KB. Without packages (had to manually edit the project file to get rid of the WINDOWS.OBJ problem) the file even got to 195 KB. So in other words: BCB really isn't a good solution for madCodeHook. I didn't expect BCB to be that bad!

Thanks for the effort!

madCodeHook supports D4-D7 and D2005. Every edition is supported, except D6 evaluation (not sure about D2005 evaluation, didn't test that yet). I'd suggest that you try to get access to a D4 or D5 standard or professional edition. Those should be *very* cheap to get. I'm not sure about the license situation, though. Maybe the standard edition must not be used for commercial projects? I believe the standard edition (D4, D5) is ok for commercial usage, but the personal edition (D6, D7) is not. Might be wrong, though. The professional edition is just fine, though. And as I said D4/5 professional should be quite cheap now.

I also see mention of Delphi 8. What is that? The only evaluation now available is Delphi 2005. Regarding the cheapest edition for commercial development, that would be the Standard edition, as you said. The Personal editions don't let you charge for your software. But Delphi 5 is the last one to have a Standard edition. Unfortunately, I could not find anyplace that still had it in stock.

So I might as well buy Delphi 2005 Professional. But for a new user license, it is US$1000! I'm not sure I can justify that. All I want is a compiler/linker that builds a simple source file! I wish it were like the Borland C++Builder, where it seems the command-line tools are free. I don't understand Borland pricing schemes. They've totally abandoned the hobbyist, and are strictly corporate now. Are you sure you want to keep using such a system for madCodeCollection?

Well, a side effect is that I'm pushing Delphi this way, which is quite welcome! But maybe a C port will come sooner or later, especially because I need to support 64bit OSs soon, and Borland doesn't offer a 64bit compiler/linker/IDE yet...

What I did yesterday is rewrite my injector to use Windows hooks instead of madCodeHook. That reduced the .exe to 40 KB. I might have to live with the .dll being 150 KB or so. The two together zip down to 88 KB. So it might be a solution, but it's a shame because now I have to implement my own IPC mechanism due to the fact the .exe no longer contains madCodeHook.

I was tempted to just use EliCZ code, which is extremely small, but does not seem as safe. I am especially concerned with uninjection; according to your comparison, he does not support safe uninjection?

I hope you decide to rewrite madCodeHook in either C or assembly. At the least, it would give BorlandC++ users an alternative, not to mention all the Microsoft VC++ users too. Moreover, I've written hook dll's as tiny as 4 KB in VC++, using Matt Pietrek tricks to reduce/remove the C startup code. Even your current Delphi .dll is an order of magnitude larger than that. I'm not sure Delphi can produce tiny dll's like this. I understand you want to promote Delphi, but at some point, it is the limiting factor here.

Cheers,
David

[madshi]

I also see mention of Delphi 8. What is that?

That's for DotNet only. So no go for hook dlls.

So I might as well buy Delphi 2005 Professional. But for a new user license, it is US$1000!

There were even some PC magazines shipping with a bundled version of D4 standard a while ago. I can get it at ebay right now for 65 Euro. That's not that much, right?

They've totally abandoned the hobbyist

Have they? The D6 personal edition was free for download for non-commercial usage!

I was tempted to just use EliCZ code, which is extremely small, but does not seem as safe. I am especially concerned with uninjection; according to your comparison, he does not support safe uninjection?

I've done the comparison a while ago. But I'm not aware that EliCZ added safe uninjecting yet. Also his solution is by far not as stable in win9x as mine, at least as far as I know. I've invented and implemented a whole lot of hacks and tricks to make the whole solution stable in all OSs.

I hope you decide to rewrite madCodeHook in either C or assembly. At the least, it would give BorlandC++ users an alternative, not to mention all the Microsoft VC++ users too.

I have quite a lot of C++ customers and nobody complained about the file size yet. Yes, madCodeHook powered MSVC++ hook dlls are not really small, but they work quite well. And these days file size is not as important as it once was.

Moreover, I've written hook dll's as tiny as 4 KB in VC++, using Matt Pietrek tricks to reduce/remove the C startup code. Even your current Delphi .dll is an order of magnitude larger than that. I'm not sure Delphi can produce tiny dll's like this. I understand you want to promote Delphi, but at some point, it is the limiting factor here.

No, it's not. The smallest possible Delphi DLL (without tricks) is 15kb. So there's really nothing to complain about. You need to understand that madCodeHook does a lot of complicated things. E.g. it implements a full disassembler to achieve good and stable API hooking. In contrast a little SetWindowsHookEx dll is almost empty. If you compare a little madCodeHook Delphi compiled hook dll to the minimum dll size in Delphi, you can see that madCodeHook itself consumes about 60kb. That wouldn't change when porting to C. Maybe you could get a hook dll in C with those special C startup tricks to 65kb, compared to 75kb for a Delphi dll, but that's it, I believe.

[dcsoft]

There were even some PC magazines shipping with a bundled version of D4 standard a while ago. I can get it at ebay right now for 65 Euro. That's not that much, right?

Have they? The D6 personal edition was free for download for non-commercial usage!

Well, I wish I had saved my old PC magazines! I think the whole situation has changed after Delphi 2005 was shipped. None of the previous Delphi versions seem to be available on borland.com anymore. They still offer install keys, but only if you already have the install CD.

I did an Internet Search and the only thing that comes close on eBay is Delphi 7 Pro for US$750. So from my point of view (first time buyer of Delphi), there seems no low cost solution available today. Where do you see D4 standard for 65 euro?

But I'm not aware that EliCZ added safe uninjecting yet. Also his solution is by far not as stable in win9x as mine, at least as far as I know. I've invented and implemented a whole lot of hacks and tricks to make the whole solution stable in all OSs.

That's what I thought... moreover, I'm not sure I would deploy anything built with EliCZ because his documentation is almost non-existant, and his website offers no support, unlike you who spoil us!

I have quite a lot of C++ customers and nobody complained about the file size yet. Yes, madCodeHook powered MSVC++ hook dlls are not really small, but they work quite well. And these days file size is not as important as it once was.

The smallest possible Delphi DLL (without tricks) is 15kb. So there's really nothing to complain about. You need to understand that madCodeHook does a lot of complicated things. E.g. it implements a full disassembler to achieve good and stable API hooking. In contrast a little SetWindowsHookEx dll is almost empty. If you compare a little madCodeHook Delphi compiled hook dll to the minimum dll size in Delphi, you can see that madCodeHook itself consumes about 60kb. That wouldn't change when porting to C. Maybe you could get a hook dll in C with those special C startup tricks to 65kb, compared to 75kb for a Delphi dll, but that's it, I believe.

Well, I sure would love my hook dll to be 65 KB, and as I have to interface with other C files, I would love to keep the hook DLL in VC. But for you, it may not be worth it to port madCodeHook from Delphi to C. I can probably live with what I have. But when you decide to add "smallest footprint" to your ever impressive feature list, I will be the first to applaud!

Now I have to go write a Mailslot IPC mechanism to replace yours....

Cheers,
David

[madshi]

I did an Internet Search and the only thing that comes close on eBay is Delphi 7 Pro for US$750. So from my point of view (first time buyer of Delphi), there seems no low cost solution available today. Where do you see D4 standard for 65 euro?

E.g. here:

http://cgi.ebay.com/ws/eBayISAPI.dll?Vi ... eName=WDVW

USD 79,99

[dcsoft]

E.g. here:

http://cgi.ebay.com/ws/eBayISAPI.dll?Vi ... eName=WDVW

USD 79,99

Hey, I missed this one! Thanks, it's a great deal. Pretty scarce too.

Cheers,
David

[dcsoft]

Just a follow up. Remember I used windows hook to replace madCodeHook Inject/Uninject in the .exe, so the .exe could be smaller? Turns out that Windows hooks had problems when injecting into AOL and Netscape... these would either crash silently or crash with an error, probably because I was injecting at an unsafe time (as soon as I noticed the frame window of the app was created).

So .exe needed to use madCodeHook's safe InjectLibrary()/UninjectLibrary(), but how to do so without making the .exe huge? I had an inspiration... since the hook .dll already had the entire madCodeHook library linked in it, why not have the .exe link with the .dll and call madCodeHook through that? Bingo, that did it!! Now the .exe is really small but can still use madCodeHook.

So that might be a way for all of you, if you are concerned about size. Don't link the madCodeHook library into both your injector .exe and the hook .dll. Just put it in the .dll and have your .exe use the .dll also.

Cheers,
David

[madshi]

Just a follow up. Remember I used windows hook to replace madCodeHook Inject/Uninject in the .exe, so the .exe could be smaller? Turns out that Windows hooks had problems when injecting into AOL and Netscape... these would either crash silently or crash with an error, probably because I was injecting at an unsafe time (as soon as I noticed the frame window of the app was created).

So .exe needed to use madCodeHook's safe InjectLibrary()/UninjectLibrary()

I didn't know that SetWindowsHookEx has stability problems. Did the problem really go away when using madCodeHook's (Un)InjectLibrary? Hey, that'd be quite a compliment. My stuff working better than the OS own APIs...

but how to do so without making the .exe huge? I had an inspiration... since the hook .dll already had the entire madCodeHook library linked in it, why not have the .exe link with the .dll and call madCodeHook through that? Bingo, that did it!! Now the .exe is really small but can still use madCodeHook.

So that might be a way for all of you, if you are concerned about size. Don't link the madCodeHook library into both your injector .exe and the hook .dll. Just put it in the .dll and have your .exe use the .dll also.

Nice idea!

One potential little problem, though: If you call UninjectLibrary, that might remove the dll from your own process, while the UninjectLibrary API is still running. To avoid this, use static linking. That way in the NT family UninjectLibrary isn't able to unload the dll from your process. However, in win9x it will still be able to do so. So you might have problem in win9x. One way to work around it is this: Copy the dll into a sub folder. This way your exe links to the dll copy in the exe's folder, while the dll in the sub folder is injected to all other processes. UninjectLibrary will then only uninject the dll in the sub folder.

[dcsoft]

I didn't know that SetWindowsHookEx has stability problems. Did the problem really go away when using madCodeHook's (Un)InjectLibrary? Hey, that'd be quite a compliment. My stuff working better than the OS own APIs...

Well, to tell the truth, it's hard to know exactly why the crashes happened. I had set up a MS Accessibility hook (SetWinEventHook) to watch for new windows of known classnames (such as the one created by AOL on startup) and when MS Accessibility reported that, that's when I did the SetWindowsHookEx() to inject the dll. Maybe that was too early, and AOL hadn't initialized yet. So I then called WaitForInputIdle() to wait for AOL to settle down before hooking. But it still crashed, but in different way. I don't think actually calling SetWindowsHookEx() caused the crash... it probably was in the DllMain of the hook dll when it called HookAPI(). But I'm not sure.

BTW, for all those interesting in hooking CreateProcess(), using SetWinEventHook() is a cheaper way of figuring out that a GUI process has started... just watch for specific windows that it creates on startup; much easier!

The crashes went away after I used your InjectLibrary(). I think you have code to inject only when safe? How do you determine that? Now, there may still be a problem; I think you've posted here that there may be a problem with calling HookAPI() when the process happens to be executing the code that you're hooking. However, in the Microsoft "Detours" documentation, they say that DllMain is a safe place to hook API's, so maybe not.

Anyway, in the testing we've done so far (not much), there haven't been any crashes so far, and the SetWindowsHookEx() crashed pretty reliably the first time we ran AOL after bootup. I hate these sporadic things!!

One potential little problem, though: If you call UninjectLibrary, that might remove the dll from your own process, while the UninjectLibrary API is still running. To avoid this, use static linking. That way in the NT family UninjectLibrary isn't able to unload the dll from your process. However, in win9x it will still be able to do so. So you might have problem in win9x. One way to work around it is this: Copy the dll into a sub folder. This way your exe links to the dll copy in the exe's folder, while the dll in the sub folder is injected to all other processes. UninjectLibrary will then only uninject the dll in the sub folder.

Thanks, but I think I'm OK... I'm not injecting into all processes (just specific processes like AOL and Netscape); specifically, I'm not calling InjectLibrary(), or UninjectLibrary() with the process of the injector .exe., so I don't think this situation would come up. Right?

Thanks,
David

[madshi]

However, in the Microsoft "Detours" documentation, they say that DllMain is a safe place to hook API's, so maybe not.

Maybe they're right. While DllMain is executed, no other threads of that process run. If DllMain creates a new thread it begins to run only after DllMain is through. So maybe DllMain is really an especially safe place to do hooking.

Thanks, but I think I'm OK... I'm not injecting into all processes (just specific processes like AOL and Netscape); specifically, I'm not calling InjectLibrary(), or UninjectLibrary() with the process of the injector .exe., so I don't think this situation would come up. Right?

Right.

[dcsoft]

Maybe they're right. While DllMain is executed, no other threads of that process run. If DllMain creates a new thread it begins to run only after DllMain is through. So maybe DllMain is really an especially safe place to do hooking.

There might still be a problem. It seems the only way to guarantee safe hooking is to make sure the API being hooked is not anywhere on the call stack of any running thread. Stopping all threads while the DLL is being injected helps, but still hooking an API that is on the call stack doesn't seem safe. When the process resumes, won't the hook screw up the API return and stack unwind?

Thanks,
David

[madshi]

Yes, it's still not 100% safe. But maybe a bit safer than outside of DllMain.

[dcsoft]

Yes, it's still not 100% safe. But maybe a bit safer than outside of DllMain.

Hmm, is there a way your hooking code could crawl the stack of each running thread and ensure the API is not on it before hooking?

If it is, it could schedule hooking for a bit later....

-- David

[madshi]

I've already thought about this some months ago. It's not enough to just check the callstack of the running threads. You also need to pause them. Because if the callstack is ok right now, that doesn't mean that it will still be alright a moment later. So I'd have to suspend all threads while installing the patch. But suspending threads is a dangerous thing and can cause its own stability problems.

[dcsoft]

I've already thought about this some months ago. It's not enough to just check the callstack of the running threads. You also need to pause them. Because if the callstack is ok right now, that doesn't mean that it will still be alright a moment later. So I'd have to suspend all threads while installing the patch. But suspending threads is a dangerous thing and can cause its own stability problems.

But the threads are already suspended, if the hooking is being done from DllMain.

If you discover it isn't safe to call HookAPI because the function is on the stack, you could uninject and try again a few milliseconds later.

[madshi]

In DllMain there could be lots of stuff which was already being executed before the HookAPI call finds out that the callstack of a thread is not ok. Simply unloading the dll is not really a good idea. What if all the other API hooks were already installed and are working just fine? And who guarantees that reinjection a few milliseconds later will solve the callstack problem? It might even be that the hook dll is written in a way that doesn't allow uninjection without crashing. I do have such a dll written myself. There's no problem with it, since I don't ever uninject it myself.