Run-Time Check Failure #0 - The value of ESP was not properl

[Protector]

Hi!

I am hooking CreateProcessA and CreateProcessW with madeCodeHook 3. The functions look like this:

Code: Select all

BOOL CreateProcessAHooked(
    __in_opt    LPCSTR lpApplicationName,
    __inout_opt LPSTR lpCommandLine,
    __in_opt    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    __in_opt    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    __in        BOOL bInheritHandles,
    __in        DWORD dwCreationFlags,
    __in_opt    LPVOID lpEnvironment,
    __in_opt    LPCSTR lpCurrentDirectory,
    __in        LPSTARTUPINFOA lpStartupInfo,
    __out       LPPROCESS_INFORMATION lpProcessInformation
	)
{
	return CreateProcessAOriginal( lpApplicationName, lpCommandLine, lpProcessAttributes, lpThreadAttributes, bInheritHandles, dwCreationFlags, lpEnvironment, lpCurrentDirectory, lpStartupInfo, lpProcessInformation );
}
BOOL WINAPI CreateProcessWHooked(
    __in_opt    LPCWSTR lpApplicationName,
    __inout_opt LPWSTR lpCommandLine,
    __in_opt    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    __in_opt    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    __in        BOOL bInheritHandles,
    __in        DWORD dwCreationFlags,
    __in_opt    LPVOID lpEnvironment,
    __in_opt    LPCWSTR lpCurrentDirectory,
    __in        LPSTARTUPINFOW lpStartupInfo,
    __out       LPPROCESS_INFORMATION lpProcessInformation
	)
{
	return CreateProcessWOriginal( lpApplicationName, lpCommandLine, lpProcessAttributes, lpThreadAttributes, bInheritHandles, dwCreationFlags, lpEnvironment, lpCurrentDirectory, lpStartupInfo, lpProcessInformation );
}

Hooking looks like this:

Code: Select all

	HookAPI("kernel32.dll", "CreateProcessA", CreateProcessAHooked, (PVOID*) &CreateProcessAOriginal);
	HookAPI("kernel32.dll", "CreateProcessW", CreateProcessWHooked, (PVOID*) &CreateProcessWOriginal);

CreateProcessAOriginal and CreateProcessWOriginal look like this:

Code: Select all

BOOL (WINAPI *CreateProcessAOriginal)(
    __in_opt    LPCSTR lpApplicationName,
    __inout_opt LPSTR lpCommandLine,
    __in_opt    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    __in_opt    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    __in        BOOL bInheritHandles,
    __in        DWORD dwCreationFlags,
    __in_opt    LPVOID lpEnvironment,
    __in_opt    LPCSTR lpCurrentDirectory,
    __in        LPSTARTUPINFOA lpStartupInfo,
    __out       LPPROCESS_INFORMATION lpProcessInformation
    );

BOOL (WINAPI *CreateProcessWOriginal)(
    __in_opt    LPCWSTR lpApplicationName,
    __inout_opt LPWSTR lpCommandLine,
    __in_opt    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    __in_opt    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    __in        BOOL bInheritHandles,
    __in        DWORD dwCreationFlags,
    __in_opt    LPVOID lpEnvironment,
    __in_opt    LPCWSTR lpCurrentDirectory,
    __in        LPSTARTUPINFOW lpStartupInfo,
    __out       LPPROCESS_INFORMATION lpProcessInformation
    );

As you can see the hooked functions just call the original functions and do nothing else. This is to keep it a very simple test case to avoid side effects from other code pieces. When I call CreateProcessW everything is fine. But when I call CreateProcessA, I get the following debug message followed by an appcrash:

Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

Any ideas? Thanks!

[madshi]

You forgot the WINAPI in your CreateProcessAHooked declaration.

[Protector]

Oh my, that's embarrassing :O

Thanks for pointing it out, I guess it's true what people say about becoming blind when reading own code.

[madshi]

No worries, that's the single most often occuring madCodeHook programming error. Double checking the calling convention is always the first thing I do...