Weird problem when intercepting fprintf

c++ / delphi package - dll injection and api hooking
Post Reply
Geert
Posts: 4
Joined: Thu Aug 12, 2004 11:54 am

Weird problem when intercepting fprintf

Post by Geert »

Hello,

I created a very simple dll that would intercept fprintf calls and print them again using vfprintf (in the end I will add functionality to that code, but for now this is outside of the scope of the problem).

See the code below.

If I remove the line #define WORKS and recompile the executable that got this dll injected into it will simply crash !
The difference between the working version and the crashing version is:
- when I use vfprintf directly it crashes
- after I Hooked vfprintf too (still don't know why I decided to try that) and then call the "next" function in the hook, it works perfectly

Altough the workaround works, I would like to understand why I need to hook vfprintf too.

Thanks,
-- Geert

Code: Select all

// ***************************************************************
//  InterceptFprintf.dll      version:  1.0   ·  date: 2004-08-12
//  -------------------------------------------------------------
//  this dll intercepts fprintf
// ***************************************************************

#include <stdio.h>
#include <stdarg.h>

#include <windows.h>
#include "madCHook.h"

#define WORKS
// ***************************************************************

int ( *ori_fprintf ) (FILE *stream, const char *format, ...);

#ifdef WORKS
int ( *ori_vfprintf) (FILE *stream, const char *format, va_list argptr);
#endif

// ***************************************************************
#ifdef WORKS
static int catch_vfprintf(FILE *stream, const char *format, va_list argptr)
{
   return(ori_vfprintf(stream, format, argptr));
}
#endif

static int catch_fprintf(FILE *stream, const char *format, ...)
{
	va_list args;
	
	va_start(args, format);

#ifdef WORKS
   return(ori_vfprintf (stream, format, args));
#else
   return(vfprintf (stream, format, args));
#endif

}


// ***************************************************************

BOOL WINAPI DllMain(HANDLE hModule, DWORD fdwReason, LPVOID lpReserved)
{
  if (fdwReason == DLL_PROCESS_ATTACH) {
    // InitializeMadCHook is needed only if you're using the static madCHook.lib
    InitializeMadCHook();

#ifdef WORKS
    HookAPI("msvcrt.dll", "vfprintf", catch_vfprintf, (PVOID*) &ori_vfprintf);
#else
    HookAPI("msvcrt.dll", "fprintf", catch_fprintf, (PVOID*) &ori_fprintf);
#endif

  } else if (fdwReason == DLL_PROCESS_DETACH)

    // FinalizeMadCHook is needed only if you're using the static madCHook.lib
    FinalizeMadCHook();

  return true;
}
Geert
Posts: 4
Joined: Thu Aug 12, 2004 11:54 am

Post by Geert »

Of course I made a type in the code (last ifdef was wrong)
Still the problem remains ...

Here is the correct (or should I say broken) code

Code: Select all

// ***************************************************************
//  InterceptFprintf.dll      version:  1.0   ·  date: 2004-08-12
//  -------------------------------------------------------------
//  this dll intercepts fprintf
// ***************************************************************

#include <stdio.h>
#include <stdarg.h>

#include <windows.h>
#include "madCHook.h"

#define WORKS
// ***************************************************************

int ( *ori_fprintf ) (FILE *stream, const char *format, ...);

#ifdef WORKS
int ( *ori_vfprintf) (FILE *stream, const char *format, va_list argptr);
#endif

// ***************************************************************
#ifdef WORKS
static int catch_vfprintf(FILE *stream, const char *format, va_list argptr)
{
   return(ori_vfprintf(stream, format, argptr));
}
#endif

static int catch_fprintf(FILE *stream, const char *format, ...)
{
	va_list args;
	
	va_start(args, format);

#ifdef WORKS
   return(ori_vfprintf (stream, format, args));
#else
   return(vfprintf (stream, format, args));
#endif

}


// ***************************************************************

BOOL WINAPI DllMain(HANDLE hModule, DWORD fdwReason, LPVOID lpReserved)
{
  if (fdwReason == DLL_PROCESS_ATTACH) {
    // InitializeMadCHook is needed only if you're using the static madCHook.lib
    InitializeMadCHook();

#ifdef WORKS
    HookAPI("msvcrt.dll", "vfprintf", catch_vfprintf, (PVOID*) &ori_vfprintf);
#endif
    HookAPI("msvcrt.dll", "fprintf", catch_fprintf, (PVOID*) &ori_fprintf);

  } else if (fdwReason == DLL_PROCESS_DETACH)

    // FinalizeMadCHook is needed only if you're using the static madCHook.lib
    FinalizeMadCHook();

  return true;
}
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

Right now I don't have a direct explanation. If both APIs would be "the same", then your not working code would end up in an endless recursion (-> crash). But on my XP the 2 APIs don't have the same address.

Are you sure that you don't need to specify a calling convention? I've not much knowledge about those msvcrt APIs.
Geert
Posts: 4
Joined: Thu Aug 12, 2004 11:54 am

Post by Geert »

Got all calling conventions set to cdecl, so that should not be an issue. I also turned off all optimizations just to be sure.
It just seems that if I don't use the vfprintf from the HookAPI call that I'm toast.

It looks like by not using the "next" vfprintf the va_args are really lost. And I think this problem is specific to hooking functions with va_args in general (but I don't know why an additional hook solves the problem).

But it gets weirder :
I noticed that I even get a different output if I install the vfprintf hook before the fprintf hook (or vice versa) !
I mean just uncomment the WORKS in the code above but remove the ifdefs for installing the hooks. Then it crashes of course because I'm still calling the (incorrect) vfprintf. Now change the order of the HookAPI calls and you will see that the output changes before the crash.

Therefore I really thing this is va_args related and somehow memory gets corrupted somewhere (was thinking the actual va_args pointer got whacked somehow - but that is just a guts feeling).

-- Geert
madshi
Site Admin
Posts: 10753
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

Hmmmm... As I said, I don't know those msvcrt functions very well. Do those APIs have a fixed parameter count? Or can you give in any number of parameters you like? In that case that would explain the problems...
Geert
Posts: 4
Joined: Thu Aug 12, 2004 11:54 am

Post by Geert »

Probably hit Murphy on the way..

Got it working now without the workaround.

I completely rebuild the workspaces from scratch to make sure avery compiler and link option was as it should be (the hint wrt calling conventions caused me to revisit the build environment).

Of course now I don't really know what the problem is, but it looked like I was mismatching /MT flag and the lib I was linking with (mvcprt.lib)
Some other minor issues might have been incorrect too...

I still find it strange that the "workaround" solved the problem, but possibly that was caused by the fact that using this workaround "forced" the code to use the correct vfprintf by referencing the one from msvcrt.dll... dunno, done enough guessing for now ...

Thanks for looking into it.
-- Geert
Post Reply