Hooking PeekMessage crash

c++ / delphi package - dll injection and api hooking
Post Reply
SpiderVenom
Posts: 8
Joined: Tue Jun 01, 2004 4:46 am
Location: New Zealand
Contact:

Hooking PeekMessage crash

Post by SpiderVenom »

I'm using madCodeHook to write a system-wide keyhook by hooking PeekMessage (GetMessage will be implemented later). My DLL code looks like this:

Code: Select all

library KeyHandler;

uses Windows, Messages, madCodeHook;

type
  TParam = record
    WParam: WPARAM;
    LParam: LPARAM;
  end;

var PeekMessageCallNext: function (lpMsg: TMsg; hWnd: HWND; wMsgFilterMin, wMsgFilterMax, wRemoveMsg: UINT): BOOL; stdcall;

function PeekMessageCallBack(lpMsg: TMsg; hWnd: HWND; wMsgFilterMin, wMsgFilterMax, wRemoveMsg: UINT): BOOL; stdcall;
var
  Param: TParam;
begin
  case lpMsg.message of
    WM_KEYFIRST..WM_KEYLAST: begin
      Param.WParam := lpMsg.wParam;
      Param.LParam := lpMsg.lParam;
      SendIpcMessage('KYHNDLR', @Param, SizeOf(Param));
    end;
  end;
  Result := PeekMessageCallNext(lpMsg, hWnd, wMsgFilterMin, wMsgFilterMax, wRemoveMsg);
end;

begin
  HookAPI(user32, 'PeekMessageA', @PeekMessageCallBack, @PeekMessageCallNext);
  HookAPI(user32, 'PeekMessageW', @PeekMessageCallBack, @PeekMessageCallNext);
end.
As soon as I inject it, it bring down every process, and the system reboots (XP). If I comment out the two HookAPI lines, everything is fine... so it seems the problem is in the DLL.

What am I doing wrong?

PS. I'm not using SetWindowsHookEx because I want to be able to inject/uninject and delete the DLL at any time. I suppose I could write a keyhook with SetWindowsHookEx, then use madRemote to execute a function in each remote process to force them to unload the DLL. Dunno if this would work though.
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

PeekMessage is called so often by the OS that it's not funny. Calling SendIpcMessage for each and every PeekMessage call will either crash the system or dramatically slow down the OS. The problem is that SendIpcMessage does cost performance. Actually right now it's rather slow. I'll improve that in a later version. It's meant to be used only for a reasonable of message per second.

Another problem is that I don't know what your IPC handler does when receiving IPC messages. If you're doing something in your IPC handler which results in PeekMessage being called you have a classic recursive deadlock.

:shock:
Post Reply