Runtime error 216 at 0017XXXX after InjectLibrarySession

[pit]

Hello,

I am using a commercial version of madCodeHook 2 (2.2l) [V2.6.2.1 installer package] with Delphi7
My client agent monitors some printing processes and consists of following parts:
A) Agent.exe (Client application in user mode)
B) Service.exe (Background service for DLL injections in system and in user mode)
C) Lib.dll (GDI hooking)

At the moment I am trying to using my agent on a terminal server (citrix). My agent application is working by the following schema:
1) Service.exe start with the system, and calls InjectLibrary(processHandle, 'Lib.dll') function for dll injection on a system start.
2) Agent.exe starts after user-session initialization.
3) Agent.exe contact Service.exe via SendIpcMessage with order to InjectDLL in user mode in to the user session id=X
4) Service.exe call InjectLibrarySession(sessionID, false, 'Lib.dll', timeOut)
5) Lib.dll starts with a process

The problem is, that in some cases the InjectLibrarySession in the step 4 returns TRUE, but the Lib.dll will not active (or it is active, but do not working well). Some target applications (such a MSWord) are works and some showing "Runtime error 216" at start and at the end (app exit). (InjectLibrarySession returns always true, regardless of whether the error occurs.)

To identify the problem area, i have created a logout to debug:
1) Service.exe => writes info about result of InjectLibrary
2) Agent.exe => writes info about executing SendIpcMessage
3) Service.exe => writes info about the result of InjectLibrarySession
4) Lib.dll => Info about dll-initialization (after begin), before HookAPI executing (after CollectHooks) and some hooking functions.

I receive following logout if a user starts an application via citrix session:
…..
08:44:02|Process „RadeRun.exe“ – DLL Initialization (after begin, before CollectHooks) [Step4]
08:44:02|Process „RadeRun.exe“ – DLL CollectHooks (after CollectHooks) [Step4]
08:44:06|Agent.exe sends a message to Service.exe [Step2]
08:44:06|Service.exe execute InjectLibrarySession => result = TRUE [Step3]
…
> 08:44:25|The problem application showing a error message (4x times)
(No logout from Lib.dll [Step 4] from the ProblemApp.exe)

If I start a another application e.g. MSWord, than i receive correct logout after the step3. (Processname, Dll start, Hook logout e.g.)
Or if I change the user-rights to full administrator, then all applications (including the ProblemApp.exe) are works fine!

I could not identify the exact position in my code, because the problem occurs after calling InjectLibrarySession (and after return true!) and before the DLL will active in the target process.

(I had used this agent already on many computers and terminal servers with different configuration, and I did not had this problem until now. Perhaps the this TS on the customer side has a special configuration. I could not reproduce this error on my testsystem.)

Any idea? Thx4help!

Regards, Alex

[madshi]

Runtime error 216 means that your hook dll is crashing, probably in DllMain or during unit initialization or finalization.

Does this problem also occur with an empty Delphi 7 hook dll (no code at all)? If not, does it occur as soon as you add madCodeHook to the uses clause, without adding any of your own code?

[pit]

Thanks for helping!

This problem does not occur if I use a empty hook dll.
I had tested following dlls (only use clause, without any code):
// => 1 (my full use string ...)
uses Windows, madCodeHook, madStrings, madKernel, SysUtils, WinSpool, PrintersNT, Registry, Classes, Math, DateUtils, NICInfo, ShellAPI, IniFiles, winSock;
// => 2
uses Windows, madCodeHook, madStrings, madKernel;
// => 3
uses Windows, madStrings, madKernel;
// => 4
uses Windows;

I think the error occurs in the DllMain, because i call log-output function directly after begin in the DllMain and in each function as well. And I could not see any log-output from this dll in the protocol file.
(But for some other processes (e.g. MS Word) this is no problem and I can see the log)
Perhaps i have the same problem in my logout function.

In my logout funciton I am using a function
1) to detect the application name via GetModuleFileNameW
2) to read a registry settings for my application via OpenKeyReadOnly (installation path)
3) to read an ini file from the installation directory
4) to write the log file via TextFile object
All functions are "secured" by try except .

What do you think, where is the problem can be located?

[madshi]

I'd recommend to not use TextFile. Use CreateFile + WriteFile + CloseHandle instead. Furthermore make sure you synchronize access to the log file (e.g. by using a critical section), because if you write to it in your hook callback functions, too, logging might run simultaneously in multiple threads. If you write to the log file from multiple processes, critical sections won't work. Then you need e.g. a named mutex. Furthermore *please* try to keep the uses clause as small as possible. Especially avoid any VCL units, if at all possible. Also avoid madKernel because it adds a lot of other units. Ideally you should only use Windows and maybe Messages and nothing else. If you really do need some stuff from other units, I'd recommend to copy the code to your hook dll. Having many units in the uses clause is not a problem in itself, but there are 2 dangers:

(1) Some of those units may add further static links to your hook dll, so loading your dll will also require further dlls to be loaded, as well. That's not good.
(2) Some of those units may have initialization/finalization sections which do bad stuff. That's very bad.

By using only Windows.pas you can guarantee that there's no hidden initialization code in your hook dll anywhere in any of the units you've used.

Ok, these rules are very strict, and if you inject your hook dll only into some specific processes and *not* system/user/session wide, then I guess you can relax a bit and maybe do some things I don't recommend. But if you do use system/user/session wide DLL injection, then you have to be ultra careful not to hurt stability, and that's why I'm setting up so strict rules of behaviour for hook dlls.

[pit]

Thank you for the fast reply!

I had added the function for writting to the file, by using of TextFile only temporarely for debugging.

I've tried to folow the rules as far as it was possible, and my dll has already works on many different systems and terminal servers properly.
Something working different on this one TS with citrix and special applications ...

I'll try now to clean up my code (remove some units from the uses) and will test the dll futher.

At the moment i am using system wide AND the user session injection on the terminal server to be sure, that my dll was really injected. I guess it is not really correct method

I'll try to swich to the user injection only and remove some not really necessary code ...

What are you think about the administrative privileges? Why do my application works on the same system if the same user gets the admin permission?

[madshi]

I've no idea. But since the problem does not occur if you have no code in your dll (other than the uses), it should be possible for you to isolate the problematic code by putting your code of your crashing hook dll back in step by step. The strict rules I described apply for user wide DLL injection, too... Of course if you have luck, your dll might still be stable, even if you don't follow those strict rules. But then, there are so many different applications out there. Some of them behave very weird. If you want to be sure that your hook dll works with really 99.99999% of all applications that exist then it's better to be extra careful. Otherwise it might work with 99.9%, but not with 99.99999% of all existing applications.