IPC Queue and Windows 10 x64

c++ / delphi package - dll injection and api hooking
Post Reply
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

IPC Queue and Windows 10 x64

Post by alfaunits »

We have found that when our DLLs are injected on Windows 10 x64, Microsoft Edge just won't start (Element not found error is reported after a few seconds, and if I try to run Edge again before that popup window, some other error occurs as well).
I am mentioning Edge, because it always reproduces the error, whereas other apps have issues on Windows 10 as well, just not always.

I've removed all of the code that deals with our hooking, i.e. I have commented out all hooks, and almost all of the code. What I have traced the issue down it is creating the IPC queue, since the error occurs when we do create the IPC Queue, and does not if we do not.

This is the actual API that sets up the IPC Queue:

Code: Select all

void __stdcall InitIpc()
{
    int     i   = 0;

    do
    {
        dwIpcQueueName  = sprintf(lpIpcQueueName, "myOwnIPCQueue#%u", i++);
    } while (!CreateIpcQueueEx(lpIpcQueueName,
                               IpcCallbackProc,
                               1,
                               0x1000));
}
If we comment out the call to this API, the crash does not occur.

Any thoughts?
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

Any ideas on this?
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

Might make sense to join this thread, because it looks like a similar or even the very same problem:

viewtopic.php?f=7&t=28135
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

I thought so, but this is one extremely cut down example that causes issues on Windoes 10 in general, Edge is just the easiest to reproduce, because it always faults.

Please try a simple DLL with just that code, or I can send you our binary with just that much.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

Ok, as a quick validation, can you please check whether the PrintMonitor demo produces the same issue for you?

http://madshi.net/PrintMonitor.zip

If it does, it's something for me to look into. If it doesn't, there seems to be something specific to your test project which makes problems (e.g. compiler or linker settings, or outdated madCodeHook version or something).
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

I don't think it works at all - it tells me the binary is not correctly signed.
The UI supposedly loads, but DebugView shows system output that says the dirver is not properly signed, and cannot be loaded. (I am running PrintMonitor64.exe As Admin)
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

I have created a dummy DLL, that solely calls the InitIPC as I said above. Here is the entire code of the DLL:

Code: Select all

#include <windows.h>
#include <stdio.h>
#include <string.h>
#include <shobjidl.h>
#include <Shlobj.h>
#include <malloc.h>
#include <crtdbg.h>
#include "madCHook.h"

DWORD   dwIpcQueueName;
char    szIpcQueueName[256];
PCHAR   lpIpcQueueName  = szIpcQueueName;

void __stdcall ITM_IpcCallback(LPCSTR   szIpc,
                               LPCVOID  lpMessage,
                               DWORD    dwMessage,
                               LPVOID   lpReply,
                               DWORD    dwReply)
{
    return;
}

void __stdcall ITM_InitIpc()
{
    int     i   = 0;

    do
    {
        dwIpcQueueName  = sprintf(lpIpcQueueName, "iFileOpIPCQueue#%u", i++);
    } while (!CreateIpcQueueEx(lpIpcQueueName,
                               ITM_IpcCallback,
                               1,
                               0x1000));
}

BOOL WINAPI DllMain(HANDLE hModule, DWORD fdwReason, LPVOID lpReserved)
{
    DWORD   dwError = 0;

    switch(fdwReason)
    {
        case DLL_PROCESS_ATTACH:
            _CrtSetDbgFlag(_CRTDBG_CHECK_ALWAYS_DF |
                           _CRTDBG_ALLOC_MEM_DF );
            InitializeMadCHook();
            ITM_InitIpc();
            break;
        case DLL_PROCESS_DETACH:
            return FALSE;
    }
    return TRUE;
}
This causes the problems with Edge loading always, and for other programs to fault. The DLL binary has absolutely no other code.

I have tried our default compiler settings (maximum debug) and also optmizations on - same result.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

alfaunits wrote:I don't think it works at all - it tells me the binary is not correctly signed.
That is weird. It seems to work fine for everybody else.

P.S: Oh wait, you're calling CreateIpcQueueEx() inside of a hook dll? That's not recommended. An IPC queue cannot be maintained without using threads, and using threads in hook dlls is kinda dangerous. See hooking rule 9:

http://help.madshi.net/HookingRules.htm

CreateIpcQueue(Ex) was designed to be called by your EXE, and the hook dll should only use SendIpcMessage(). In theory calling CreateIpcQueue(Ex) in a hook dll could also work, but it's really not recommended. Is there no other way to achieve your goal?
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

Hmm.
We do create an IPC queue in our apps, but that queue serves for DLLs to send messages to us - whereas, we need to send some callback messages (during file copy, for progress) to each DLL individually (so we cannot use the IPC queue created by the apps for the reply)

I have tinkered with doing my own file mapping, but the IPC queue made this so much faster.

Why would it cause a total hang though, only on W10?

And about the Print Monitor: the app does not present any errors, it loads the UI without errors. The DebugView output shows the error.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

In the other thread we found out that Edge already runs into problems if we just enable a few privileges that are disabled by default. It seems Edge is very delicate. Creating an IPC queue requires changing privileges, creating threads and stuff, which is a LOT of things for a hook dll, which is supposed to be unobtrusive and barely noticeable for the target application.

The IPC solution for EXE -> DLL communication I usually recommend is to use CreateGlobalFileMapping in your EXE, and then OpenFileMapping in your hook dll. This way your EXE can write information to the shared memory section, and your hook dll can read it when needed. This is very low footprint IPC and doesn't require any dramatic changes to the target application.
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

We would need a separate thread for this also :(
Since the callback would be during processing of SendIpcMessage (from our DLL to our app, our app would then need to send a callback, and NOT finish processing the IPC message still)

What is done:
- DLL hooks file copy operations (for non-security reasons)
- DLL sends an IPC message to our apps (one of them only for each copy/move)
- Our app(s) need to callback for each chunk of data being copied.

The last step cannot be done via replying to the message..
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

Ok. You can still do this via a memory mapped file, it's just a bit more complicated. Basically you'd do a SendIpcMessage from the DLL to the application, without waiting for a reply. Then you'd poll a global memory mapped file (created by the application) for a reply from the application. Or if you don't want to poll, you can use an event to allow the application to signal to the DLL that the memory mapped file was updated. Polling is probably not a big problem, though, if the reply is expected quickly. In any case, all of this should be possible to do from just the thread which called your hook callback function.
alfaunits
Posts: 21
Joined: Sat Apr 09, 2011 9:41 pm

Re: IPC Queue and Windows 10 x64

Post by alfaunits »

Hmm, that seems plausable! It will require lots of rewrite, but it is possible.
madshi
Site Admin
Posts: 10754
Joined: Sun Mar 21, 2004 5:25 pm

Re: IPC Queue and Windows 10 x64

Post by madshi »

It would definitely be good for long-time overall stability, so I think it would be a good investment.
Post Reply