Page 1 of 1

MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Tue Sep 22, 2015 11:08 am
by EaSy
Hi,
we have a new problem with hooking shell32!CFileOperation::SetOperationFlags on W8 and higher x64 version using new MCH 3.1.10. The problem is that the hooked orig function returns E_INVALIDARG instead of S_OK. The only difference from the previous version is the bugfix "(2) fixed: some conflicts with other hook libraries (x64)". Are you able to fix this or need more info? Thx.

PP

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Tue Sep 22, 2015 2:52 pm
by madshi
That's quite surprising, to be honest. Is this only with x64, or does it happen with x86, too? Could you provide a disasm of the to-be-hooked function before and after it's hooked? And maybe also a disasm of the trampoline, if that's not too much effort. That would be a great help - thank you!

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Wed Sep 23, 2015 8:17 am
by EaSy
orig

Code: Select all

0:087> uf shell32!CFileOperation::SetOperationFlags
SHELL32!CFileOperation::SetOperationFlags:
00007ffc`dfcf8dc8 f6c203          test    dl,3
00007ffc`dfcf8dcb 0f85ec9e2f00    jne     SHELL32!CFileOperation::SetOperationFlags+0xe (00007ffc`dfff2cbd)

SHELL32!CFileOperation::SetOperationFlags+0x5:
00007ffc`dfcf8dd1 4883c1f0        add     rcx,0FFFFFFFFFFFFFFF0h
00007ffc`dfcf8dd5 eb09            jmp     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation (00007ffc`dfcf8de0)

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation:
00007ffc`dfcf8de0 899158030000    mov     dword ptr [rcx+358h],edx
00007ffc`dfcf8de6 89514c          mov     dword ptr [rcx+4Ch],edx
00007ffc`dfcf8de9 f6c240          test    dl,40h
00007ffc`dfcf8dec 0f85d19e2f00    jne     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0x11 (00007ffc`dfff2cc3)

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0xe:
00007ffc`dfcf8df2 33c0            xor     eax,eax
00007ffc`dfcf8df4 c3              ret

SHELL32!CFileOperation::SetOperationFlags+0xe:
00007ffc`dfff2cbd b857000780      mov     eax,80070057h
00007ffc`dfff2cc2 c3              ret

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0x11:
00007ffc`dfff2cc3 81ca00000820    or      edx,20080000h
00007ffc`dfff2cc9 89514c          mov     dword ptr [rcx+4Ch],edx
00007ffc`dfff2ccc e92161d0ff      jmp     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0xe (00007ffc`dfcf8df2)
3.1.9

Code: Select all

0:038> uf 00007ffc`e1ed0000
SHELL32!CFileOperation::SetOperationFlags+0x5:
00007ffc`dfcf8dd1 4883c1f0        add     rcx,0FFFFFFFFFFFFFFF0h
00007ffc`dfcf8dd5 eb09            jmp     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation (00007ffc`dfcf8de0)

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation:
00007ffc`dfcf8de0 899158030000    mov     dword ptr [rcx+358h],edx
00007ffc`dfcf8de6 89514c          mov     dword ptr [rcx+4Ch],edx
00007ffc`dfcf8de9 f6c240          test    dl,40h
00007ffc`dfcf8dec 0f85d19e2f00    jne     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0x11 (00007ffc`dfff2cc3)

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0xe:
00007ffc`dfcf8df2 33c0            xor     eax,eax
00007ffc`dfcf8df4 c3              ret

SHELL32!CFileOperation::SetOperationFlags+0xe:
00007ffc`dfff2cbd b857000780      mov     eax,80070057h
00007ffc`dfff2cc2 c3              ret

SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0x11:
00007ffc`dfff2cc3 81ca00000820    or      edx,20080000h
00007ffc`dfff2cc9 89514c          mov     dword ptr [rcx+4Ch],edx
00007ffc`dfff2ccc e92161d0ff      jmp     SHELL32!CFileOperation::SetOperationFlagsWithoutValidation+0xe (00007ffc`dfcf8df2)

00007ffc`e1ed0000 ff2500000000    jmp     qword ptr [00007ffc`e1ed0006]

00007ffc`e1ee0000 f6c203          test    dl,3
00007ffc`e1ee0003 0f85b42c11fe    jne     SHELL32!CFileOperation::SetOperationFlags+0xe (00007ffc`dfff2cbd)

00007ffc`e1ee0009 e9c38de1fd      jmp     SHELL32!CFileOperation::SetOperationFlags+0x5 (00007ffc`dfcf8dd1)
3.1.10

Code: Select all

0:108> uf 00007ffc`e1ee0000
00007ffc`e1ee0000 ff2500000000    jmp     qword ptr [00007ffc`e1ee0006]

00007ffc`e1ef0000 f6c203          test    dl,3
00007ffc`e1ef0003 50              push    rax
00007ffc`e1ef0004 48b8bd2cffdffc7f0000 mov rax,offset SHELL32!CFileOperation::SetOperationFlags+0xe (00007ffc`dfff2cbd)
00007ffc`e1ef000e 48870424        xchg    rax,qword ptr [rsp]
00007ffc`e1ef0012 c3              ret
3.1.10 version seems to be broken.

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Wed Sep 23, 2015 8:35 am
by madshi
Oh, I see, thanks. The new code obviously didn't check if it's a conditional (e.g. jne) or unconditional (jmp) jump, and treated both the same. I'm sorry about that. Will fix it urgently and release a new build.

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Wed Sep 23, 2015 8:40 am
by EaSy
thx

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Fri Oct 09, 2015 1:22 pm
by madshi
Should be fixed in this build:

http://madshi.net/madCollectionBeta.exe (installer 2.7.12.2)

Can you confirm?

Re: MCH 3.1.10 and shell32!CFileOperation::SetOperationFlags

Posted: Mon Oct 12, 2015 11:54 am
by EaSy
It is working.