HookAPI not working at all

c++ / delphi package - dll injection and api hooking
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

HookAPI not working at all

Post by Jaque »

Ok, what I'm trying to do is very simple. I use iTunesMobileDevice.dll in my application (part of iTunes). Under certain circumstances it will call TerminateProcess. I need to hook TerminateProcess in my process so that I can stop my application from just vanishing without communicating a problem to the user.

I have added the evaluation version of madCodeHook 2 into my application and the iTunes dll still manages to execute TerminateProcess despite me redirecting it. So your hooking library doesn't appear to work at all, which I doubt is really the case. What could I be doing wrong? Moreover, how can the iTunes DLL get around the hook? I've looked at the DLL in IDA and it doesn't appear to do anything special, it just calls GetCurrentProcess and passes the result to TerminateProcess.


DLL SOURCE:

type
TTerminateProcess = function (hProcess: THandle; uExitCode: UINT): BOOL; stdcall;

var
uNextHook: TTerminateProcess;


function TerminateProcessOverride(hProcess: THandle; uExitCode: UINT): BOOL; stdcall;
begin
Result := True;
end;

begin
HookAPI('kernel32.dll', 'TerminateProcess', @TerminateProcessOverride, uNextHook);
end;
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

Update: If I call TerminateProcess from within my own code, my hook proc gets called, and the process doesn't terminate.

So somehow iTunesMobileDevice.dll is getting the real address of TerminateProcess and evading the hook.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

Please in the Delphi debugger check if the disasm of TerminateProcess starts with a JMP instruction after you did the HookAPI() call. Please only check afterwards, because the disasm sometimes remembers the old code and doesn't refresh it. It's possible that the evaluation system somehow fails, but usually you should get a MessageBox if that happens. So I'm not sure what's going on. You did copy the madCHook.dll into the system32 folder? Could be that your anti-virus company is blocking things somehow? The old madCodeHook 2.x version has been misused by some malware writers, because of that it's sometimes classified as "dangerous" by security software. No such problem with madCodeHook 3.x, fortunately.

In newer OSs most kernel32.dll APIs are really implemented in kernelbase.dll. Maybe the iTunes dll calls the API from kernelbase? Would be somewhat strange, but not impossible.

We're definitely talking about the iTunes dll terminating your own process, correct?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

Cross-post.

Your "the real address" comment suggests a misunderstanding of the hooking concept. madCodeHook modifies "the real address". The iTunes dll is supposed to call "the real address" and the hook should still work just fine.

Try hooking kernelbase.dll. Or alternatively hook ntdll.dll's NtTerminateProcess, which is the lowest API level in user land.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

>>Please in the Delphi debugger check if the disasm of TerminateProcess starts with a JMP instruction after you did the HookAPI() call. <<

Can't really do that. I'm running Windows 8.1, but since your eval won't work in 8.1, I'm having to test this in a Win 7 VM.

>>You did copy the madCHook.dll into the system32 folder? <<

Yes

>>Could be that your anti-virus company is blocking things somehow?<<

It's all disabled.

>>Maybe the iTunes dll calls the API from kernelbase? <<

When I look at the iTunes DLL in IDA, it clearly shows the import as coming from kernel32.dll - there's no reference to kernelbase.dll or ntdll.dll, at least pertaining to terminating the process.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

>>We're definitely talking about the iTunes dll terminating your own process, correct?<<

Yes, I don't care about other processes.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

I've upload a test dll here:

http://madshi.net/HookTerminateProcessTest.rar

Code: Select all

library Project1;

uses Windows, madCodeHook;

type
  TTerminateProcess = function (hProcess: THandle; uExitCode: UINT): BOOL; stdcall;

var
  uNextHook: TTerminateProcess;

function TerminateProcessOverride(hProcess: THandle; uExitCode: UINT): BOOL; stdcall;
begin
  MessageBox(0, 'TerminateProcess called', 'info', 0);
  result := uNextHook(hProcess, uExitCode);
end;

begin
  HookAPI('kernel32.dll', 'TerminateProcess', @TerminateProcessOverride, @uNextHook);
end.
Compiled with madCodeHook 3.x. Does it work? If not, you can do the disasm test in your win8.1 machine.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

I tried hooking NtTerminateProcess, and I am getting a callback, but it's causing my app to crash with a windows error.

I'm not sure I've got the correct prototype:

I've tried
TNtTerminateProcess = function (AProcess, AExitStatus: cardinal): cardinal; stdcall;
and
TNtTerminateProcess = function (AProcess, AExitStatus: cardinal): cardinal; cdecl;
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

Should be "function NtTerminateProcess(processHandle: THandle; exitStatus: dword) : HRESULT; stdcall". Important is that both your callback definition and the "next" function variable have the correct parameters and calling convention. If it still crashes, show me the full code, not just a fragment.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

Still crashing.

Code: Select all

library HookLib;

uses
  System.SysUtils,
  System.Classes,
  WinApi.Windows,
  madCodeHook,
  UnitCode_SelfProtectionTypes;

{$R *.res}

const
  HOOK_MEM_FILENAME = '{7C2DF810-72FB-48E0-8757-C5B40053A995}';

type
  TNtTerminateProcess = function (hProcess: THandle; uExitStatus: DWORD): HRESULT; stdcall;

  TTerminateProcessNotify = procedure (AProcess: THandle; AExitCode: Cardinal); stdcall;

  THookData = record
    Hooked: boolean;
    CanTerminateProcess: boolean;
    TerminateProcessCallback: TTerminateProcessNotify;
    TerminateProcessNextHook: TNtTerminateProcess;
  end;
  PHookData = ^THookData;

var
  uHookData: PHookData;
  uMemFile: THandle;

procedure MemShared();
begin
  uMemFile := OpenFileMapping(FILE_MAP_ALL_ACCESS, False, HOOK_MEM_FILENAME);
  if uMemFile = 0 then
    uMemFile := CreateFileMapping($FFFFFFFF, nil, PAGE_READWRITE, 0, SizeOf(THookData), HOOK_MEM_FILENAME);
  if uMemFile <> 0 then
  begin
    uHookData := MapViewOfFile(uMemFile, FILE_MAP_ALL_ACCESS, 0, 0, 0);
    uHookData^.Hooked := False;
    uHookData^.CanTerminateProcess := False;
    uHookData^.TerminateProcessNextHook := nil;
    uHookData^.TerminateProcessCallback := nil;
  end;
end;

function TerminateProcessOverride(hProcess: THandle; uExitCode: DWORD): HRESULT; stdcall;
begin
  if Assigned(uHookData^.TerminateProcessCallback) then
    uHookData^.TerminateProcessCallback(hProcess, uExitCode);
  if Assigned(uHookData^.TerminateProcessNextHook) and uHookData^.CanTerminateProcess then
    Result := uHookData^.TerminateProcessNextHook(hProcess, uExitCode)
  else
    Result := $C0000022; // return access denied
end;

procedure BeginHook; stdcall;
begin
  if not uHookData^.Hooked then
  begin
    if not HookAPI('ntdll.dll', 'NtTerminateProcess', @TerminateProcessOverride, @uHookData^.TerminateProcessNextHook) then
      raise Exception.Create('NtTerminateProcess HookAPI failed');
    uHookData^.Hooked := True;
  end;
end;

procedure EndHook; stdcall;
begin
  if uHookData^.Hooked then
  begin
    if not UnhookAPI(@uHookData^.TerminateProcessNextHook) then
      raise Exception.Create('NtTerminateProcess UnhookAPI failed');
    uHookData^.Hooked := False;
  end;
end;

procedure SetCanTerminateProcess(AValue: boolean); stdcall;
begin
  uHookData^.CanTerminateProcess := AValue;
end;

procedure SetTerminateProcessCallback(const AValue: TTerminateProcessNotify); stdcall;
begin
  uHookData^.TerminateProcessCallback := AValue;
end;

exports
  BeginHook,
  EndHook,
  SetCanTerminateProcess,
  SetTerminateProcessCallback;

begin
  MemShared;
end.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

I've changed TerminateProcessOverride to

Code: Select all

function TerminateProcessOverride(hProcess: THandle; uExitCode: DWORD): HRESULT; stdcall;
begin
  MessageBox(0, 'TerminateProcessOverride', 'WTF', MB_ICONWARNING);
  Result := 0;
end;
I get the message box, indicating the redirection is there, but then process still terminates!!! I don't get it.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

Well, what can I say. I guess you need to do some debugging to find out what's going on there.

Have you tried the dll I compiled for you? That one should allow you to debug the process with the Delphi debugger in Windows 8.1.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

No, with your test project the redirect never gets invoked - same as mine.

Like I said earlier, the madcodehook eval app won't run in Win 8.1, so I'm having to use a win 7 VM to test with - as a result, I can't run my code in the debugger.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI not working at all

Post by madshi »

Alright, here's a test dll compiled with madCodeHook 3 which hooks NtTerminateProcess:

http://madshi.net/HookNtTerminateProcessTest.rar

I've tested it in Delphi 7 running on win8.1 x64 with the following code:

Code: Select all

  LoadLibrary('D:\Desktop\hookTest\project1.dll');
  TerminateProcess(GetCurrentProcess, 0);
  MessageBox(0, 'hook succeeded', 'info', 0);
Works fine. First the hook dll shows a MessageBox, then the MessageBox "hook succeeded" appears. So the TerminateProcess call was successfully hooked and blocked. Since this test dll was compiled with the commercial madCodeHook 3.0 edition, it will run on your win8.1, too, so you can use this dll to do debugging in Delphi, if needed.
Jaque
Posts: 18
Joined: Sat May 24, 2008 12:56 pm

Re: HookAPI not working at all

Post by Jaque »

Thanks

I ran it in the debugger and,

ntdll:7717FC50 ntdll_NtTerminateProcess proc near
ntdll:7717FC50 jmp off_71AE001E
ntdll:7717FC50 ntdll_NtTerminateProcess endp

yes, the first (and only) instruction of NtTerminateProcess is a JMP

Still trying to track down how the process still terminates despite being redirected.
Post Reply