Page 1 of 4

It's about use of CreateIpcQueue() in Windows10.

Posted: Fri May 12, 2017 5:23 am
by icckitagawa
Hello, madshi.
I'm not good at English, so I have a translation software help. It's hoped to be transmitted.

I'm using madcodehook4.0.2.

When using CreateIpcQueue(), I think I met memory leak.
I find this by Win10-32bit and don't bring by Win7-32bit,Win7-64bit.
Win10-64bit is being investigated.

The latest patch is applied to respective Windows by an echelon on May 11 (Japan time).
Win10-32bits of a problem are a Pro edition and 1703 or version OS build 15063.296.
The relevant cord is as follows.

Code: Select all

//Make IPCQueue

//madchook4.0
	b_que = CreateIpcQueue("Devctl_DVCSendType", (PIPC_CALLBACK_ROUTINE)&CB_Devctl_DVCSendType,0,1,4096 );
	if ( !b_que ) {
	  DispLog("[Error]CreateIpcQueueEx Devctl_DVCSendType");
	}

//madchook4.0
	b_que = CreateIpcQueue("Devctl_DVCStatType", (PIPC_CALLBACK_ROUTINE)&CB_Devctl_DVCStatType,0,1,4096 );
	if ( !b_que ) {
	  DispLog("[Error]CreateIpcQueueEx Devctl_DVCStatType");
	}


//IPCQueue Callback
void WINAPI CB_Devctl_DVCSendType(
  LPCSTR  pIpc,
  LPCVOID pMessageBuf,
  DWORD   dwMessageLen,
  LPVOID  pAnswerBuf,
  DWORD   dwAnswerLen,
  LPVOID  pContext
  ) 
{

	memcpy( pAnswerBuf,(void *)&gDVCSendType,dwAnswerLen );

	return;
}

void WINAPI CB_Devctl_DVCStatType(
  LPCSTR  pIpc,
  LPCVOID pMessageBuf,
  DWORD   dwMessageLen,
  LPVOID  pAnswerBuf,
  DWORD   dwAnswerLen,
  LPVOID  pContext
  ) 
{

	memcpy( pAnswerBuf,(void *)&gDVCStatType[0],dwAnswerLen );

	return;
}
When a comment out does a part in CreateIpcQueue() of this code, memory leak stops.

Memory leak was observed at ProcessExplorer and VMMap.
When the performance of a property of a process of a problem is confirmed in ProcessExplorer:.
Virtual Size in Virtual Memory is about the percentage of 100 K a second, and it increases.
When it's done for a while, a process is suspended.

Size 4K of [Shareable] will be made for confirmation in VMMap in quantities.

I'm moving in Win7, so however perplexed it be whether to handle.
Beta (2.8.2.2) was tried, but it wasn't improved.

If you can have advice of improvement, it's lucky.

Thank you very much.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Fri May 12, 2017 8:18 am
by madshi
Could you create a small test project with which I could reproduce this problem on my PC?

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Fri May 12, 2017 9:38 am
by icckitagawa
Thank you very much for your answer.

I see. I'll write the sample code to which a point was narrowed down and try.

If there is development, it'll be reported.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Mon May 15, 2017 4:50 am
by icckitagawa
I confirmed this phenomenon using "HookProcessTermination" of the sample project.
We built the sample HookProcessTermination with VisualStudioExpress2012 as it is.

The reproduction method is as follows.
1. Run HookProcessTermination with the latest Windos 10-32 bit.
2. Monitor HookProcessTermination32.exe with ProcessExplorer. Always display Virtual Size of Virtual Memory.
3. Start "Notepad.exe". Notepad.exe will then appear in the ProcessExplorer tree.
4. Exit Notepad.exe with ProcessExplorer's menu "Kill Process".
5. The ProcessExplorer dialog "Are you sure you want to kill notepad.exe?" Is displayed. Remember the value of Virtual Size at this time.
6. Click "OK" in the message of 5. Then, "HookProcessTermination32.exe" will display "May the process procexpexe terminate the following process? Notepad.exe".
7. Do not terminate Notepad.exe by selecting "No" for this message.
8. The message "Error terminating process; Access denied" is displayed from ProcessExplorer. Click the "OK" button to close it.
9. The value of Virtual Size at this time is memorized and compared with the value at 5. As a result, it is 4 kByte more.


As you can see from this result, if you send using SendIpcMessage, it seems that the process that created Queue with CreateIpcQueue will generate 4 KByte memory leak for each communication.

We corrected it to execute 4 Kbyte memory leak in one communication and execute SendIpcMessage more than 100 times, and observed it the same way, but a memory leak of 404 KByte was confirmed.
The correction outline of the code is as follows.

--------------

Code: Select all

[HookProcessTermination.cpp]

int WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR, int)
{
  InitializeMadCHook();

  // create an ipc queue, through which our dll can contact us
  CHAR arrCh [MAX_PATH];

  //Make Dummy IPCQueue
  wsprintf(arrCh, "HookProcessTermination_Dummy%u", GetCurrentSessionId());
  CreateIpcQueue(arrCh, HandleProcessTerminationRequest_Dummy);


  wsprintf(arrCh, "HookProcessTermination%u", GetCurrentSessionId());
  if (CreateIpcQueue(arrCh, HandleProcessTerminationRequest))

------------
[HookTerminateAPIs.cpp]

BOOL IsAllowed(HANDLE hProcess)
// ask the user whether the current process may terminate the specified process
{
					:
					:
					:
      // we're an application running in a specific session
      // let's contact the HookProcessTermination application instance
      // which runs in the same session as we do
      session = GetCurrentSessionId();

		int i;
		for(i=0;i<100;i++){
	    wsprintf(arrChA, "HookProcessTermination_Dummy%u", session);
		SendIpcMessage(arrChA,
                        &tr,     sizeof(tr),        // our message
                        &result, sizeof(result));   // the answer
		}


    // contact our application, which then will ask the user for confirmation
    // hopefully there's an instance running in the specified session
    wsprintf(arrChA, "HookProcessTermination%u", session);
    if (!SendIpcMessage(arrChA,

--------------

I ran the same program on Windows 7 - 32 bit and measured in the same way, but I could not confirm the memory leak.


Since the project file used for this experiment is "HookProcessTermination" of the sample project, I do not think it is necessary to send it, but please specify it if necessary.

I hope to know what is causing this problem early.
I am pleased to be able to borrow the power.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Mon May 15, 2017 7:14 am
by madshi
I don't have a win10 32bit VM right now. Does this also happen in win10 64bit?

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Mon May 15, 2017 8:42 am
by icckitagawa
Thank you for your reply.

Whether the same thing happens with win10-64bit, it is thought that it will happen.

In order to make HookProcessTermination of the sample project work with 64 bit, the EV signature is required for the driver, but we can not sign it now due to our convenience.

Instead, when you observe the memory leak of CreateIpcQueue with a program that already operates with EV signature, memory leak is occurred in win10-64bit in the same way.

However, there are two types of DLLs to be injected, 32 bit and 64 bit, both of which are executing SendIpcMessage. It is not possible to distinguish which communication causes leakage.
(Or, both may be affected)

I hope it will be helpful.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Thu May 18, 2017 12:49 am
by icckitagawa
Hello.

Sorry, what happened to this memory leak issue?

We imagine that madshi is busy dealing with other problems.
But I'm glad if you also look at this issue.

The problem of this time is 4 kbyte memory leak for IPC communication once.
Our project is injected into all programs of the login session and communicates at a frequency of once a second.
Therefore, a memory leak of about 100 KByte per second occurs, and the process executing CreateIpcQueue () is stopped at a very early timing.

It is impossible to lower the monitoring frequency or not by doing IPC communication according to the specification of the project.
We think that it is impossible and borrowing the power of madshi in order to solve the problem.

I would be very happy if you could tell me about solutions and workarounds.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Thu May 18, 2017 6:50 am
by madshi
It's on my to do list, and I'll come to it "soon", please have patience.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Thu May 18, 2017 8:20 am
by icckitagawa
I am sorry to hurry.

We know that the Madshi team is dealing with many difficult problems.
Of course it is OK as far as possible, please make a good product.


Well, I thought that I could manage this problem at present.

Fortunately, fortunately DestroyIpcQueue () seems to release the IPC queue together as a memory leak as well.

I used this to duplicate the IPC queue, so I made alternate DestroyIpcQueue and CreateIpcQueue every minute.
The SendIpcMessage on the DLL side is configured to communicate with the connected party of the duplicated IPC queue.


By this, it has not become a fundamental solution, but it began to work.
I am going to do aging test this evening for overnight to investigate the degree of memory leak and stability.

If the result is good, shipping of our project is done for now for now.
However, since the code is quite "bad," I expect the official modified version.


Thanks to Microsoft, I think it is hard work, but please do its best.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Fri May 19, 2017 8:16 am
by icckitagawa
I carried out the idea I said before.

The result was not very good.

Execution of DestroyIpcQueue certainly releases the IPC queue together with memory leaks.

However, if you execute it several times, it will not return from DestroyIpcQueue.

I do not know the timing when this phenomenon occurs.
(It occurred when repeating DestroyIpcQueue and CreateIpcQueue about 5,000 times in continuous test.
However, since it may occur earlier, I think that the number is not the cause)

Once DestroyIpcQueue stops processing, IPC queues with the same name can not be created.
Since DestroyIpcQueue does not return processing, I threaded it and ran and gave 10 seconds of grace.
After timeout, the thread terminates with TerminateThread.
After that, DestroyIpcQueue will be executed, but the return value will be 0, then you can not create an IPC queue with the same name on CreateIpcQueue.


I thought it was a good idea, but it is regrettable without being stable.
However, as it is incomplete at the moment, it is a method of using the IPC queue without causing a memory leak, so I would like to adopt it.

Everyone of the experts who are watching this will be happy to tell you if you have a good idea.

I am looking forward to a modified version of the official.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Tue May 23, 2017 4:33 am
by icckitagawa
Hello, madshi.

I'd like to know the schedule of corrections to explain to my customers.
I used to answer "soon" before, but I want to know the specific announcement timing of the modified module.

My customers also understand the circumstances and think that it is best to wait for the correction of madshi.
However, my client wants to know the timing of the modified version in order to explain the product release plan to the end user.


Even if you look at other threads, you can see that madshi is difficult and copes with many problems.
I also know that such a question is painful when you are doing your best. I'm sorry.

I want you to make a good product. I do not want it in a hurry to forcibly. I would like you to tell me when it will be announced.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Tue May 23, 2017 6:30 am
by madshi
Hopefully this week.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Wed May 24, 2017 1:35 am
by icckitagawa
Thank you, madshi.

I told your prospect to my customer.

I feel bad that I stopped your hand on account of me.
I expect your good work.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Wed Jun 07, 2017 12:20 am
by icckitagawa
Hello, madshi.

Two weeks have passed since I inquired.
When is release of a modified version?

I inquired to a support mail about it, but there are no answers. I think anxiously.

Please tell me an operational perspective.

Thank you.

Re: It's about use of CreateIpcQueue() in Windows10.

Posted: Tue Jun 13, 2017 5:09 pm
by madshi
I'm sorry for having taken so very long. I've now upgraded my development PC to Windows 10, which made analyzing this problem much easier. Here's what I found:

1) I've removed most of the IPC server side code, but the problem still occurs.
2) Sending IPC messages up to 240 bytes doesn't seem to trigger the problem, but 241 bytes or more does, although my code doesn't care at all if it's 240 or 241 bytes.
3) The lost memory comes back in exactly the moment when I call "CloseHandle(IpcPortHandle)".
4) My code is just a wrapper around some native OS APIs like "NtCreatePort" etc.

All this combined makes me 99.9% sure that the problem is not in my code. It seems that the OS itself produces this memory problem when we send IPC messages with more than 240 bytes. I don't know why. And I don't think there's anything I can do about it, because the problem is very likely outside of my code.

Hmmm... The problem seems to be limited to the "Virtual Size" report in Process Explorer, which is probably not even allocated memory, but just "reserved" memory? It could still be a problem, though.

The only thing coming to my mind right now is that you could try to limit the size of your IPC messages to max 240 bytes. Or alternatively we could try to find a way for you to close and reopen the IPC queue regularly without crashing.

What do you think?