However, the callback I registered is not called even though HookAPI succeeded and DeviceEject.exe called CM_Request_Device_Eject_ExW.
Just in case, when I created a test program and called CM_Request_Device_Eject_ExW directly in the program, the callback was called normally through API hooking.
It seems that hooking is not working only for C:\Windows\System32\DeviceEject.exe, but I would like to know the cause.
The test was conducted on Windows 10 64bit 22H2 (19045.2604),
DeviceEject.exe was executed by calling it in the following form from cmd.
"C:\WINDOWS\System32\DeviceEject.exe" 0 "Instance path of currently connected USB" 1
e.g.) "C:\WINDOWS\System32\DeviceEject.exe" 0 "USB\VID_0781&PID_5567\4C530001300816116481" 1
2. I used HookAPI at the DLLMain entry point to hook the CM_Request_Device_Eject_ExW of CfgMgr32.dll, and as a result, TRUE was returned.
3. I used a separate program called API Monitor to confirm that CfgMgr32.dll's CM_Request_Device_Eject_ExW is called in DeviceEject.exe.
I also confirmed that the DLL I created was injected into DeviceEject.exe, and the API hook was performed, and then CM_Request_Device_Eject_ExW was called.
Are there any other things to check?
Also, how do you know that the hook callback function was truly not called? I've had a case reported where someone thought the hook callback function was not called because their call to OutputDebugString didn't work, for some reason, for it was the OutputDebugString which failed, not the hook. Maybe you can use some other method to double check instead of OutputDebugString? E.g. create an empty file in a folder that you have NTFS write access to.