HookAPI failing

c++ / delphi package - dll injection and api hooking
Post Reply
blackpaw
Posts: 33
Joined: Mon Nov 05, 2007 1:08 am

HookAPI failing

Post by blackpaw »

I'm getting a FALSE from HookAPI when hooking on Win 7 64 bit:

Code: Select all

	void DoHook()
	{
		InitThunk((TMFP) &CInitHook::OurInitializePrintMonitor2, this, 2);

		FARPROC thunk = GetThunk();
		if (thunk == NULL)
			LogItem(LOG_INFO, NULL, L"Thunk Failed");
		else
		{
			LogItem(LOG_INFO, NULL, L"Thunk Succeeded");
			BOOL rc = HookAPI(dll.c_str(), "InitializePrintMonitor2", thunk, (PVOID*) &OrigInitializePrintMonitor2);
			if (rc)
				LogItem(LOG_INFO, NULL, L"Hook Succeeded");
			else
				LogItem(LOG_INFO, NULL, L"Hook Failed");
		};
You'll note that I'm using a thunk, to get a closure from a Class method.

Same code works fine when not using the thunk, just static functions but I need my "this" pointer :(

Any idea why HookAPI fails? nothing in the docs.

Code for the thunk below:

Code: Select all

template <class T>
class CAuxThunk
{
	// stub "code" is no longer the class itself
   BYTE m_thunk[1024]; // fixed; don't know size required apriori!
	// for 3 arguments the size required is under 200 bytes

public:
  typedef void (T::*TMFP)();

  // need an extra parameter, # of real arguments in the callback
  void InitThunk(TMFP method, T* pThis, int nArgs = 3)
  {
	  ATLASSERT(static_cast< CAuxThunk<T>* >(this) == pThis); // not really required as argument?
	  union { void* (*func)(); TMFP method; } addr;
	  addr.method = (TMFP)method;
	  ATLASSERT(sizeof(TMFP)==sizeof( void* (*)() )); // shouldn't be virtual
#if defined(_DEBUG) && defined(_WIN64)
	  // @@@ 64 bit compiler creates TMFP pointers that are 16 bytes long (!) even without virtuals
	  ATLASSERT(sizeof(TMFP)==sizeof(ULONGLONG)*2);
	  union {
		  struct {ULONGLONG a, b;};
		  TMFP method;
	  } ck_;
	  ck_.method = method;
	  ATLASSERT(0==ck_.b); // no useful information in high pointer?
#endif

	  // pass "this" as the first fixed argument
	  // that will put it on the stack instead of ECX, as should be in x64
	  void *fixed[] = { pThis };
	  size_t thunk_size = vbind(addr.func, nArgs+1, m_thunk, 0, fixed, 1);
	  ATLASSERT(thunk_size < sizeof(m_thunk));
	  
	  unsigned long old;
	  VirtualProtect(m_thunk, thunk_size, PAGE_EXECUTE_READWRITE, &old);
	  FlushInstructionCache(GetCurrentProcess(), m_thunk, thunk_size);
  }

  FARPROC GetThunk() const { return (FARPROC)(void*)m_thunk; }
};

Thanks.
blackpaw
Posts: 33
Joined: Mon Nov 05, 2007 1:08 am

Re: HookAPI failing

Post by blackpaw »

NB, I added call to test my thunk:

Code: Select all

			TOrigInitializePrintMonitor2 test = (TOrigInitializePrintMonitor2) thunk;
			LogItem(LOG_INFO, NULL, L"Testing Funk");
			test(NULL, 0);
Logging showed it invoked the method call correctly with the right "this" pointer.
blackpaw
Posts: 33
Joined: Mon Nov 05, 2007 1:08 am

Re: HookAPI failing

Post by blackpaw »

Issue is resolved - oddly HookAPI always seems to return false. And Something else I was doing was breaking the hook.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: HookAPI failing

Post by madshi »

Are you sure that HookAPI() really returns false, but still works correctly? That would be very weird! If that's really the case, could you create a small test project for me? Thanks!
Post Reply