SSL Hooking

[TJ]

Hello,

I'm trying to debug a 3rd-party app that uses SSL to send/receive data. I really only care about the receiving part. So I tried to hook SSL_read using madCodeHook. Initially everything looks okay: the HookAPI function returns true, and the nextHook parameter is set to some reasonable-looking value. However, when the target app makes a call to SSL_read and my callback function calls the "real" SSL_read, the real SSL_read always returns 1 and some junk character data is in pData. At that point the target app stops being able to communicate with the server. The code I'm using is:

Code: Select all

int (WINAPI* SSL_readNextHook)(void* pSSL, void* pData, int length);

int WINAPI SSL_readCallback (
  void* pSSL,
  void* pData,
  int length
   ) {

  int retval=SSL_readNextHook(pSSL,pData,length);
  FILE* logFile=fopen("c:\\ssl_read.log","a+");

  fprintf(logFile,"SSL_read returned %i; Data: ",retval);
  for(int charCnt=0;charCnt<retval;charCnt++)
    fprintf(logFile,"%c",((char*)(pData))[charCnt]);

  fclose(logFile);

  return retval;
}

Any ideas what I might be doing wrong?

[madshi]

Not sure. Are you sure it's really WINAPI? Maybe it's CDECL?

[TJ]

I'm not sure. How would I find out if it's WINAPI or CDECL or...?

[madshi]

Check the SSL header file. Or simply try to compile your hook DLL with CDECL to see whether it fixes the problem.