GFI FaxMaker FaxServer service triggers BSOD on the server indirectly while terminating the orphan document conversion processes

Versions / Builds Affected

GFI FaxMaker 2013 SR3

Status

Open

TT / JIRAID

746

How to Identify

THIS IS A FOLLOW UP of TT 620. Please advise what we should do next: CURRENT: Development concluded that the server crash was not caused by GFI FaxMaker FaxServer process, although Microsoft pointed fmserver.exe as the reason for the crash and NOW also the fax card manufacturer Gerdes. Here is what customer said about the analysis of Gerdes (fax card manufacturer): I am contacting you regarding the blue screens and dump files from the past. GERDES finally responded with their analysis of the dump files. However, they cannot find anything pointing at their driver. Here is the German quote: Alle 3 Dumps zeigen exakt dieselbe Fehlerursache, es handelt sich um eine CRITICAL_OBJECT_TERMINATION Exception im Kontext von fmserver.exe. Einen Verweis auf den PrimuX Treiber finde ich in den Dumps nicht. Es ist möglich, dass ein Treiber z.B. durch das Überschreiben zufälliger Speicherbereiche zu Instabilitäten und Abstürzen führt, allerdings passiert dies dann in der Regel an ganz unterschiedlichen Stellen und nicht, wie in Ihrem Fall, immer genau an derselben Stelle. Dies deutet daraufhin, dass fmserver.exe auch die Ursache ist. I am now without any options. Dell (hardware), Microsoft (OS), Gerdes (ISDN Driver/Card) and GFI (Software) are all telling me, that they are not responsible for this problem. However, servers are still crashing and it is always fmserver.exe causing the issue. Do you have any idea how to move forward? HISTORY: Recently, the customer experienced several server crashes. We provided Microsoft with the created memory dump files and they found the reason for the crash. It seems like GFI Faxmaker is causing the server to crash. BSOD screenshot attached Full dumo files uploaded by the customer on the FTP server: ftp://ftp.gfisoftware.com/support/ aldi_2014.05.14_fullmemory.zip aldi_2014.05.15_fullmemory.zip Question: Why does FM FaxServer process cause the crash? Analyse: =========== Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64 Product: Server, suite: TerminalServer SingleUserTS Built by: 7601.18229.amd64fre.win7sp1_gdr.130801-1533 Machine Name: "501CSL02" Kernel base = 0xfffff800`0184e000 PsLoadedModuleList = 0xfffff800`01a916d0 Debug session time: Wed Mar 26 03:40:48.562 2014 (UTC + 2:00) System Uptime: 1 days 3:24:50.494 .bugcheck Bugcheck code 000000F4 Arguments 00000000`00000003 fffffa80`08226890 fffffa80`08226b70 fffff800`01bca0d0 !cpuinfo CP F/M/S Manufacturer MHz PRCB Signature MSR 8B Signature Features 0 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 1 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 2 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 3 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 4 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 5 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 6 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe 7 6,45,7 GenuineIntel 2800 0000070d00000000 21193ffe Cached Update Signature 0000070d00000000 Initial Update Signature 0000070d00000000 2014.03.26 501csl02 BSOD\MEMORY.DMP Debug session time : 2014-03-26 01:40:48.000 Kernel-Only Dump, AMD x64, 64-bit Addresses, PAE=n/a, 3GB=n/a. Windows Server 2008 R2, Service Pack 1. ComputerName = 501CSL02. ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* CRITICAL_OBJECT_TERMINATION (f4) A process or thread crucial to system operation has unexpectedly exited or been terminated. Several processes and threads are necessary for the operation of the system; when they are terminated (for any reason), the system can no longer function. Arguments: Arg1: 0000000000000003, Process Arg2: fffffa8008226890, Terminating object Arg3: fffffa8008226b70, Process image file name Arg4: fffff80001bca0d0, Explanatory message (ascii) 0: kd> !thread THREAD fffffa8009072540 Cid 08e4.0ec8 Teb: 000000007ee8a000 Win32Thread: 0000000000000000 RUNNING on processor 0 Not impersonating DeviceMap fffff8a002b94250 Owning Process fffffa80097b4240 Image: fmserver.exe Attached Process N/A Image: N/A Wait Start TickCount 6326273 Ticks: 0 Context Switch Count 109301 IdealProcessor: 3 UserTime 00:00:03.744 KernelTime 00:00:01.263 Win32 Start Address 0x00000000004263a0 Stack Init fffff8800716cc70 Current fffff8800716c290 Base fffff8800716d000 Limit fffff88007167000 Call 0 Priority 9 BasePriority 8 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr : Args to Child : Call Site fffff880`0716c9c8 fffff800`01c51d92 : 00000000`000000f4 00000000`00000003 fffffa80`08226890 fffffa80`08226b70 : nt!KeBugCheckEx fffff880`0716c9d0 fffff800`01bfe1db : ffffffff`ffffffff fffffa80`09072540 fffffa80`08226890 fffffa80`097b4240 : nt!PspCatchCriticalBreak+0x92 fffff880`0716ca10 fffff800`01b7dec4 : ffffffff`ffffffff 00000000`00000001 fffffa80`08226890 00000000`00000008 : nt!PspTerminateAllThreads+0x80c27 fffff880`0716ca60 fffff800`018c2e13 : fffffa80`08226890 fffff880`00000000 fffffa80`09072540 00000000`00000b50 : nt!NtTerminateProcess+0xf4 fffff880`0716cae0 00000000`77c8157a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0716cae0) 00000000`1398e808 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x77c8157a 0: kd> !object fffffa8008226890 Object: fffffa8008226890 Type: (fffffa80069a2de0) Process ObjectHeader: fffffa8008226860 (new version) HandleCount: 6 PointerCount: 121 0: kd> !process fffffa8008226890 PROCESS fffffa8008226890 SessionId: 0 Cid: 0268 Peb: 7fffffd5000 ParentCid: 0220 DirBase: 21c481000 ObjectTable: fffff8a0014885c0 HandleCount: 88. Image: wininit.exe VadRoot fffffa80082250b0 Vads 63 Clone 0 Private 453. Modified 0. Locked 2. DeviceMap fffff8a000008b30 Token fffff8a00146e600 ElapsedTime 1 Day 03:24:43.605 UserTime 00:00:00.000 KernelTime 00:00:00.015 QuotaPoolUsage[PagedPool] 98016 QuotaPoolUsage[NonPagedPool] 20496 Working Set Sizes (now,min,max) (1284, 50, 345) (5136KB, 200KB, 1380KB) PeakWorkingSetSize 1300 VirtualSize 48 Mb PeakVirtualSize 51 Mb PageFaultCount 1386 MemoryPriority BACKGROUND BasePriority 13 CommitCharge 540 THREAD fffffa8008229060 Cid 0268.026c Teb: 000007fffffde000 Win32Thread: fffff900c00df8f0 WAIT: (UserRequest) UserMode Non-Alertable fffffa8008240f50 NotificationEvent Not impersonating DeviceMap fffff8a000008b30 Owning Process fffffa8008226890 Image: wininit.exe Attached Process N/A Image: N/A Wait Start TickCount 616 Ticks: 6325657 (1:03:24:40.881) Context Switch Count 357 IdealProcessor: 2 LargeStack UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000ffe06290 Stack Init fffff88003786c70 Current fffff880037867c0 Base fffff88003787000 Limit fffff8800377d000 Call 0 Priority 15 BasePriority 15 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Kernel stack not resident. Child-SP RetAddr Call Site fffff880`03786800 fffff800`018b95d2 nt!KiSwapContext+0x7a fffff880`03786940 fffff800`018ca99f nt!KiCommitThreadWait+0x1d2 fffff880`037869d0 fffff800`01bbb55e nt!KeWaitForSingleObject+0x19f fffff880`03786a70 fffff800`018c2e13 nt!NtWaitForSingleObject+0xde fffff880`03786ae0 00000000`77c812fa nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`03786ae0) 00000000`0028fb18 00000000`00000000 0x77c812fa THREAD fffffa8008240060 Cid 0268.02b8 Teb: 000007fffffd6000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Alertable fffffa800823fd80 SynchronizationTimer fffffa800823fef0 SynchronizationTimer fffffa800822bb30 ProcessObject fffffa800824db30 ProcessObject fffffa8008252b30 ProcessObject fffffa800823fc10 SynchronizationTimer Not impersonating DeviceMap fffff8a000008b30 Owning Process fffffa8008226890 Image: wininit.exe Attached Process N/A Image: N/A Wait Start TickCount 12235 Ticks: 6314038 (1:03:21:39.624) Context Switch Count 11 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x0000000077c4aef0 Stack Init fffff88002ba5c70 Current fffff88002ba4e80 Base fffff88002ba6000 Limit fffff88002ba0000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Kernel stack not resident. Child-SP RetAddr Call Site fffff880`02ba4ec0 fffff800`018b95d2 nt!KiSwapContext+0x7a fffff880`02ba5000 fffff800`018c60ea nt!KiCommitThreadWait+0x1d2 fffff880`02ba5090 fffff800`01bbb88f nt!KeWaitForMultipleObjects+0x272 fffff880`02ba5350 fffff800`01bbbbfe nt!ObpWaitForMultipleObjects+0x294 fffff880`02ba5820 fffff800`018c2e13 nt!NtWaitForMultipleObjects+0xe5 fffff880`02ba5a70 00000000`77c8186a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`02ba5ae0) 00000000`00bff8d8 00000000`00000000 0x77c8186a THREAD fffffa8008f92b50 Cid 0268.17e0 Teb: 000007fffffdc000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable fffffa800822dac0 QueueObject Not impersonating DeviceMap fffff8a000008b30 Owning Process fffffa8008226890 Image: wininit.exe Attached Process N/A Image: N/A Wait Start TickCount 6325857 Ticks: 416 (0:00:00:06.489) Context Switch Count 1472 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x0000000077c4fbf0 Stack Init fffff8800804dc70 Current fffff8800804d680 Base fffff8800804e000 Limit fffff88008048000 Call 0 Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site fffff880`0804d6c0 fffff800`018b95d2 nt!KiSwapContext+0x7a fffff880`0804d800 fffff800`018cb1e3 nt!KiCommitThreadWait+0x1d2 fffff880`0804d890 fffff800`01bad267 nt!KeRemoveQueueEx+0x323 fffff880`0804d950 fffff800`018aede6 nt!IoRemoveIoCompletion+0x47 fffff880`0804d9e0 fffff800`018c2e13 nt!NtWaitForWorkViaWorkerFactory+0x285 fffff880`0804dae0 00000000`77c82bba nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0804dae0) 00000000`00eaf718 00000000`00000000 0x77c82bba \\\ >>> Ready Threads /// Cur Bas zContext Kernel Time User Time Elapsed Ticks Q-TSA Thread Wait Waiting | Overview | Start PROC Image Name No. Thread Id Pri Pri Switches d:hh:mm:ss.ttt d:hh:mm:ss.ttt d:hh:mm:ss.ttt RrBLsrl State Reason Notes Waiting On Function | Function | Function CPU 00: 08E4 fmserver.exe 1 !t fffffa80`09c00060 052c 8 8 9660756 .156 .514 0 -r----- Ready Delay 08E4 fmserver.exe 2 !t fffffa80`09c46640 1238 8 8 3983 0 .062 0 -r----- Ready Delay 08E4 fmserver.exe 3 !t fffffa80`09ba9640 047c 8 8 68173 .124 1.497 0 -r----- Ready Delay 0BF8 tmcsvc.exe 4 !t fffffa80`097da760 0fa8 8 8 905763 0 0 0 -r----- Ready Delay 0004 System 5 !t fffffa80`069e8b50 0008 1 0 1124752 38.719 0 0 -r---r- Ready FreePg. CPU 01: No Ready Threads CPU 02: No Ready Threads CPU 03: No Ready Threads CPU 04: No Ready Threads CPU 05: No Ready Threads CPU 06: No Ready Threads CPU 07: No Ready Threads 5 Ready Threads found 0: kd> !process 0268 0 Searching for Process with Cid == 268 PROCESS fffffa8008226890 SessionId: 0 Cid: 0268 Peb: 7fffffd5000 ParentCid: 0220 DirBase: 21c481000 ObjectTable: fffff8a0014885c0 HandleCount: 88. Image: wininit.exe 0: kd> !process 0220 0 Searching for Process with Cid == 220 PROCESS fffffa800b8dfb30 SessionId: 0 Cid: 0220 Peb: 7efdf000 ParentCid: 08e4 DirBase: 12972d000 ObjectTable: fffff8a007eba1d0 HandleCount: 121. Image: shex.exe 0: kd> !process 08e4 0 Searching for Process with Cid == 8e4 PROCESS fffffa80097b4240 SessionId: 0 Cid: 08e4 Peb: 7efdf000 ParentCid: 0d34 DirBase: 1e0887000 ObjectTable: fffff8a00adf1200 HandleCount: 752. Image: fmserver.exe 0: kd> !process 0d34 0 Searching for Process with Cid == d34 PROCESS fffffa8009a55060 SessionId: 0 Cid: 0d34 Peb: 7efdf000 ParentCid: 02c4 DirBase: 1162f6000 ObjectTable: fffff8a002429f90 HandleCount: 59. Image: fmservic.exe Looks like fmserver.exe is causing the system to bugcheck by terminating wininit.exe purposely. Example for Exchange: Exchange process msexchangerepl.exe called into terminate process on wininit.exe. This is a design feature of exchange to crash the machine when disk IO's are taking too long. Maybe we have although a design feature in fmserver.exe. 0x000000F4 =========== 2690750 "0x000000F4" Stop error when you stop a PowerShell script in Windows 7 or in Windows Server 2008 R2 2876900 FIX: Stop Error "0x000000F4" occurs when an application creates a process in a cross-session manner in Windows 7 or Windows Server 2008 R2 2675806 "0x000000F4" Stop error when you try to start a ClickOnce application in Windows Vista, in Windows 7, in Windows Server 2008 or in Windows Server 2008 R2 2775511 An enterprise hotfix rollup is available for Windows 7 SP1 and Windows Server 2008 R2 SP1 Notice: After this update is installed, you must install update 2732673 to fix a regression issue in the Rdbss.sys file. To do this, install update 2732673 from the following Microsoft Knowledge Base article: 2732673 "Delayed write failed" error message when .pst files are stored on a network file server that is running Windows Server 2008 R2 After this update is installed, you must install update 2728738 to fix a regression issue in the Profsvc.dll file. To do this, install update 2728738 from the following Microsoft Knowledge Base article: 2728738 You experience a long logon time when you try to log on to a Windows 7-based or a Windows Server 2008 R2-based client computer that uses roaming profiles After this update is installed, you must install update 2878378 re-released on November 11, 2013 to fix a regression issue in the Advapi32.dll file. To do this, install update 2878378 from the following Microsoft Knowledge Base article: 2878378 OpsMgr 2012 or OpsMgr 2007 R2 generates a "Heartbeat Failure" message and then goes into a greyed out state in Windows Server 2008 R2 SP1 The updated versions of Rdbss.sys, Profsvc.dll, and Advapi32.dll are also available on the Microsoft Update Catalog after you enter each of the previously mentioned KB article numbers into the search field on the Microsoft Update Catalog website. The following are pre-populated URLS for the previously mentioned fixes: http://catalog.update.microsoft.com/v7/site/Search.aspx?q=2732673 http://catalog.update.microsoft.com/v7/site/Search.aspx?q=2728738

Workaround / Fix Details

A hotfix is available for the potential bug Hotfix will only provided by the PSG directly. Please escalate this case.

Required Actions

A hotfix is available for the potential bug Hotfix will only provided by the PSG directly. Please escalate this case.