SEH in CallbackFunc

c++ / delphi package - dll injection and api hooking
Post Reply
nikara
Posts: 4
Joined: Tue Sep 14, 2004 3:13 pm

SEH in CallbackFunc

Post by nikara »

I think it would be better not to use C runtime functions in hook callback functions, since the thread which calls the hooked API may be created by CreateThread(), not by _beginthreadex(). Is this right?

However, I'd like to use __try/__catch of SEH or try/catch of C++ for error handling in callback functions. Are these possible?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

>> I think it would be better not to use C runtime functions in hook callback functions, since the thread which calls the hooked API may be created by CreateThread(), not by _beginthreadex(). Is this right?

Yes, I think so.

>> However, I'd like to use __try/__catch of SEH or try/catch of C++ for error handling in callback functions. Are these possible?

I'm not sure what the C++ compiler makes out of that SEH stuff, so I can't really answer that question. If you want an anwer from me about that, please post the assembler code here to which C++ compiled the SEH stuff. Then I can probably say whether it's safe to use it or not.
nikara
Posts: 4
Joined: Tue Sep 14, 2004 3:13 pm

Post by nikara »

Thank you for your reply, madshi.

This is a SEH sample code.

Code: Select all

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
BOOL APIENTRY DllMain( HANDLE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
    return TRUE;
}

BOOL WINAPI CallbackProcW(LPCWSTR szStr)
{
	WCHAR a[10];
	__try
	{
		lstrcpyW(a, szStr); // overrun buffer?
	}
	__except (EXCEPTION_EXECUTE_HANDLER)
	{
		a[0] = NULL;
	}
	return TRUE;
}
I compiled by VC++ .NET 2003 with the following compile option.
It is necessary to link with LIBMT.lib to avoid unresolved external symbol errors
(__except_list, __except_handler3, __load_config_used)
compile
/O2 /G5 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /D "_USRDLL" /D "SEHTEST_EXPORTS" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /FD /EHsc /MT /FAcs /Fa"Release/" /Fo"Release/" /Fd"Release/vc70.pdb" /W3 /nologo /c /Wp64 /Zi /TP

link
/OUT:"Release/SehTest.dll" /INCREMENTAL:NO /NOLOGO /DLL /NODEFAULTLIB /DEBUG /PDB:"Release/SehTest.pdb"
/SUBSYSTEM:WINDOWS /OPT:REF /OPT:ICF /ENTRY:"DllMain" /IMPLIB:"Release/SehTest.lib" /MACHINE:X86
LIBCMT.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib
nikara
Posts: 4
Joined: Tue Sep 14, 2004 3:13 pm

Post by nikara »

And this is the assembler code.
Would you please advise me?

Code: Select all

; Listing generated by Microsoft (R) Optimizing Compiler Version 13.10.3077 

	TITLE	.\SehTest.cpp
	.386P
include listing.inc
if @Version gt 510
.model FLAT
else
_TEXT	SEGMENT PARA USE32 PUBLIC 'CODE'
_TEXT	ENDS
_DATA	SEGMENT DWORD USE32 PUBLIC 'DATA'
_DATA	ENDS
CONST	SEGMENT DWORD USE32 PUBLIC 'CONST'
CONST	ENDS
_BSS	SEGMENT DWORD USE32 PUBLIC 'BSS'
_BSS	ENDS
$$SYMBOLS	SEGMENT BYTE USE32 'DEBSYM'
$$SYMBOLS	ENDS
$$TYPES	SEGMENT BYTE USE32 'DEBTYP'
$$TYPES	ENDS
_TLS	SEGMENT DWORD USE32 PUBLIC 'TLS'
_TLS	ENDS
;	COMDAT _IsEqualGUID
_TEXT	SEGMENT PARA USE32 PUBLIC 'CODE'
_TEXT	ENDS
;	COMDAT _==
_TEXT	SEGMENT PARA USE32 PUBLIC 'CODE'
_TEXT	ENDS
;	COMDAT _DllMain@12
_TEXT	SEGMENT PARA USE32 PUBLIC 'CODE'
_TEXT	ENDS
;	COMDAT ?CallbackProcW@@YGHPBG@Z
_TEXT	SEGMENT PARA USE32 PUBLIC 'CODE'
_TEXT	ENDS
sxdata	SEGMENT DWORD USE32 'SXDATA'
sxdata	ENDS
FLAT	GROUP _DATA, CONST, _BSS
	ASSUME	CS: FLAT, DS: FLAT, SS: FLAT
endif

INCLUDELIB LIBCMT
INCLUDELIB OLDNAMES

PUBLIC	_DllMain@12
; Function compile flags: /Ogty
; File c:\documents and settings\user1\my documents\visual studio projects\sehtest\sehtest.cpp
;	COMDAT _DllMain@12
_TEXT	SEGMENT
_hModule$ = 8						; size = 4
_ul_reason_for_call$ = 12				; size = 4
_lpReserved$ = 16					; size = 4
_DllMain@12 PROC NEAR					; COMDAT

; 8    :     return TRUE;

  00000	b8 01 00 00 00	 mov	 eax, 1

; 9    : }

  00005	c2 0c 00	 ret	 12			; 0000000cH
_DllMain@12 ENDP
_TEXT	ENDS
PUBLIC	?CallbackProcW@@YGHPBG@Z			; CallbackProcW
EXTRN	__imp__lstrcpyW@8:NEAR
EXTRN	__except_handler3:NEAR
EXTRN	__except_list:DWORD
;	COMDAT CONST
CONST	SEGMENT
$T18787	DD	0ffffffffH
	DD	FLAT:$L18783
	DD	FLAT:$L18784
; Function compile flags: /Ogty
CONST	ENDS
;	COMDAT ?CallbackProcW@@YGHPBG@Z
_TEXT	SEGMENT
_a$ = -44						; size = 20
__$SEHRec$ = -24					; size = 24
_szStr$ = 8						; size = 4
?CallbackProcW@@YGHPBG@Z PROC NEAR			; CallbackProcW, COMDAT

; 12   : {

  00000	55		 push	 ebp
  00001	8b ec		 mov	 ebp, esp
  00003	6a ff		 push	 -1
  00005	68 00 00 00 00	 push	 OFFSET FLAT:$T18787
  0000a	68 00 00 00 00	 push	 OFFSET FLAT:__except_handler3
  0000f	64 a1 00 00 00
	00		 mov	 eax, DWORD PTR fs:__except_list
  00015	50		 push	 eax
  00016	64 89 25 00 00
	00 00		 mov	 DWORD PTR fs:__except_list, esp
  0001d	83 ec 1c	 sub	 esp, 28			; 0000001cH
  00020	53		 push	 ebx
  00021	56		 push	 esi
  00022	57		 push	 edi
  00023	89 65 e8	 mov	 DWORD PTR __$SEHRec$[ebp], esp

; 13   : 	WCHAR a[10];
; 14   : 	__try

  00026	c7 45 fc 00 00
	00 00		 mov	 DWORD PTR __$SEHRec$[ebp+20], 0

; 15   : 	{
; 16   : 		lstrcpyW(a, szStr); // overrun buffer?

  0002d	8b 45 08	 mov	 eax, DWORD PTR _szStr$[ebp]
  00030	50		 push	 eax
  00031	8d 4d d4	 lea	 ecx, DWORD PTR _a$[ebp]
  00034	51		 push	 ecx
  00035	ff 15 00 00 00
	00		 call	 DWORD PTR __imp__lstrcpyW@8
  0003b	eb 09		 jmp	 SHORT $L18792
$L18783:
$L18793:

; 17   : 	}
; 18   : 	__except (EXCEPTION_EXECUTE_HANDLER)

  0003d	b8 01 00 00 00	 mov	 eax, 1
$L18791:
$L18785:
  00042	c3		 ret	 0
$L18784:
  00043	8b 65 e8	 mov	 esp, DWORD PTR __$SEHRec$[ebp]
$L18792:
  00046	c7 45 fc ff ff
	ff ff		 mov	 DWORD PTR __$SEHRec$[ebp+20], -1

; 19   : 	{
; 20   : 		a[0] = NULL;
; 21   : 	}
; 22   : 	return TRUE;

  0004d	b8 01 00 00 00	 mov	 eax, 1

; 23   : }

  00052	8b 4d f0	 mov	 ecx, DWORD PTR __$SEHRec$[ebp+8]
  00055	64 89 0d 00 00
	00 00		 mov	 DWORD PTR fs:__except_list, ecx
  0005c	5f		 pop	 edi
  0005d	5e		 pop	 esi
  0005e	5b		 pop	 ebx
  0005f	8b e5		 mov	 esp, ebp
  00061	5d		 pop	 ebp
  00062	c2 04 00	 ret	 4
?CallbackProcW@@YGHPBG@Z ENDP				; CallbackProcW
_TEXT	ENDS
END
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

Hmmm... Well, it's really difficult to say. The asm code shown above it ok. It depends on what "__except_handler3" does internally. I'd say: If you can get along without the try..catch then do that. If you really need it, then do some tests with normal win32 threads to see whether there are any problems.
nikara
Posts: 4
Joined: Tue Sep 14, 2004 3:13 pm

Post by nikara »

Thank you for quick reply.

I'll try to remove try/catch blocks as much as possible,
and use try/catch block only in unavoidable parts with sufficient test as you said.
Thank you very much for your comments.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Post by madshi »

Often you can get around using try..catch frames by adding additional checks or by using IsBadReadPtr or such things. At least that's what I'm usually doing...
Post Reply