Subject Re: [firebird-support] Re: Terminate Long Running Transaction
Author Vlad Horsun
> Here is the trace..

Lets look inside


> *----> State Dump for Thread Id 0x964 <----*

This thread is main (first) thread. Here you can see WinMain,
StartServiceCtrlDispatcher and SetServiceStatus functions.
this thread do nothing with database

> *----> Stack Back Trace <----*
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\WINDOWS\system32\ADVAPI32.dll -
> *** WARNING: Unable to verify checksum for C:\Program
> Files\Firebird\bin\fbserver.exe
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\WINDOWS\system32\kernel32.dll -
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 0022f7ac 77f795ab 000000a4 0022f870 0000021a ntdll!KiFastSystemCallRet
> 0022f7d8 77f7943c 000000a4 0022f870 0000021a ADVAPI32!
> SetServiceStatus+0x243
> 0022f84c 77fb2ec9 000000a4 0022f870 0000021a ADVAPI32!
> SetServiceStatus+0xd4
> 0022fa90 0055efcc 005e1f90 77e66404 0024252d ADVAPI32!
> StartServiceCtrlDispatcherA+0xf7
> 0022ff18 0053c618 00400000 00000000 0024252d fbserver!WinMain+0x1ac
> (CONV: stdcall)
> 0022ffc0 77e523e5 00000000 00000000 7ffd6000 fbserver!
> WinMainCRTStartup+0x185 (CONV: cdecl)
> 0022fff0 00000000 0053c493 00000000 78746341 kernel32!
> IsProcessorFeaturePresent+0x9e


> *----> State Dump for Thread Id 0xed4 <----*

This thread is Service Control Manager thread - you can found CNTL_main_thread
function. It is coordinated windows service with SCM and do nothing with datbaase

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 00b3ff70 77e6ba42 000000cc ffffffff 00000000 ntdll!KiFastSystemCallRet
> 00b3ff84 00564fdb 000000cc ffffffff 00249380 kernel32!
> WaitForSingleObject+0x12
> 00b3ffa4 77f79348 00000001 0024938c 00000000 fbserver!
> CNTL_main_thread+0xeb (FPO: [EBP 0x00b3ffb8] [2,2,0]) (CONV: stdcall)
> 00b3ffb8 77e6608b 00249380 00000000 00000000 ADVAPI32!
> ConvertStringSidToSidW+0x255
> 00b3ffec 00000000 77f79327 00249380 00000000 kernel32!
> GetModuleFileNameA+0xeb



> *----> State Dump for Thread Id 0xc6c <----*

Can't say much about this thread but it is just do Sleep

> *----> Stack Back Trace <----*
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\Program Files\Firebird\bin\MSVCR71.dll -
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 00d3ff60 77e424c1 00000064 00000000 00d3ffb8 ntdll!KiFastSystemCallRet
> 00d3ff70 00418b34 00000064 00000000 006149a0 kernel32!Sleep+0xf
> 00d3ff84 7c349565 00000000 00000000 00000000 fbserver!
> ThreadPriorityScheduler::schedulerMain+0x24 (FPO: [1,0,0]) (CONV:
> stdcall)
> 00d3ffb8 77e6608b 00614908 00000000 00000000 MSVCR71!endthreadex+0xa0
> 00d3ffec 00000000 7c3494f6 00614908 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0xcdc <----*

This is TCP server thread. It is listen on IP socket

> *----> Stack Back Trace <----*
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\WINDOWS\System32\mswsock.dll -
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\WINDOWS\system32\WS2_32.dll -
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 01317bb0 71b23a52 000000fc 0000010c 00000000 ntdll!KiFastSystemCallRet
> 01317ca0 71c0470c 00000400 005e7e64 00000000 mswsock+0x3a52
> 01317cf0 00506ade 00000400 005e7e64 00000000 WS2_32!select+0xa1
> 01317e24 00506c8d 0091ac70 04022658 00910638 fbserver!
> select_wait+0x10e (FPO: [EBP 0x0131feec] [1,68,0]) (CONV: cdecl)
> 01317e34 00509d43 0091ac70 01317ed8 04022000 fbserver!
> select_multi+0x8d (FPO: [4,0,0]) (CONV: cdecl)
> 01317e48 00564641 01317ed8 04022000 01317ed0 fbserver!
> rem_port::select_multi+0x13 (FPO: [3,0,0]) (CONV: thiscall)
> 0131feec 0055ec7a 0091ac70 0000003a 0091edc8 fbserver!
> SRVR_multi_thread+0xd1 (CONV: cdecl)
> 0131ff50 00418a3e 00000000 00000000 00404b39 fbserver!
> inet_connect_wait_thread+0x7a (FPO: [1,20,0]) (CONV: stdcall)
> 0131ff5c 00404b39 00000000 00614ad0 00614908 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 0131ff84 7c349565 0091edc8 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 0131ffb8 77e6608b 00614908 00000000 00000000 MSVCR71!endthreadex+0xa0
> 0131ffec 00000000 7c3494f6 00614908 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0x4d8 <----*

This is Named Pipe server thread. It is listen on server's named pipe end

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 0151fd48 0050907b 00000114 00000000 0091ab54 ntdll!KiFastSystemCallRet
> 0151fee4 0055ecd6 005ed7c8 00000000 0151ff04 fbserver!
> WNET_connect+0x18b (FPO: [EBP 0x0151ff84] [4,96,0]) (CONV: cdecl)
> 0151ff50 00418a3e 00000000 00000000 00404b39 fbserver!
> wnet_connect_wait_thread+0x36 (FPO: [1,20,0]) (CONV: stdcall)
> 0151ff5c 00404b39 00000000 00614b68 00614908 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 0151ff84 7c349565 0091abb0 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 0151ffb8 77e6608b 00614908 00000000 00000000 MSVCR71!endthreadex+0xa0
> 0151ffec 00000000 7c3494f6 00614908 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0xc74 <----*

This is XNET server thread. It is also wait some event

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 0171fe8c 77e6ba42 00000130 ffffffff 00000000 ntdll!KiFastSystemCallRet
> 0171fea0 00504dd8 00000130 ffffffff 00919470 kernel32!
> WaitForSingleObject+0x12
> 0171fed8 00504f68 0171ff04 0000003a 0055ed73 fbserver!
> connect_server+0x68 (CONV: cdecl)
> 0171fee4 0055ed73 00000000 00000000 0171ff04 fbserver!
> XNET_connect+0x28 (FPO: [4,0,0]) (CONV: cdecl)
> 0171ff50 00418a3e 00000000 00000000 00404b39 fbserver!
> xnet_connect_wait_thread+0x33 (FPO: [1,20,0]) (CONV: stdcall)
> 0171ff5c 00404b39 00000000 00614c00 00614908 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 0171ff84 7c349565 009194e4 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 0171ffb8 77e6608b 00614908 00000000 00000000 MSVCR71!endthreadex+0xa0
> 0171ffec 00000000 7c3494f6 00614908 00000000 kernel32!
> GetModuleFileNameA+0xeb



> *----> State Dump for Thread Id 0x42c <----*

This is cache_writer thread. It is sleep now

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 033efdd8 77e6109d 00000001 033efe1c 00000001 ntdll!KiFastSystemCallRet
> 033efdf4 00403ac1 00000001 033efe1c 00000001 kernel32!
> WaitForMultipleObjects+0x18
> 033efe58 0042fa49 00000001 033eff3c 033eff58 fbserver!
> ISC_event_wait+0x91 (FPO: [EBP 0x033eff50] [6,16,0]) (CONV: cdecl)
> 033eff50 00418a3e 00000001 00000000 00404b39 fbserver!
> cache_writer+0x249 (CONV: stdcall)
> 033eff5c 00404b39 00000000 00614d60 00614ef0 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 033eff84 7c349565 00913de0 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 033effb8 77e6608b 00614ef0 00000000 00000000 MSVCR71!endthreadex+0xa0
> 033effec 00000000 7c3494f6 00614ef0 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0xb04 <----*

This is another cache_writer thread (we have it per database). It is sleep too

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 0367fdd8 77e6109d 00000001 0367fe1c 00000001 ntdll!KiFastSystemCallRet
> 0367fdf4 00403ac1 00000001 0367fe1c 00000001 kernel32!
> WaitForMultipleObjects+0x18
> 0367fe58 0042fa49 00000001 0367ff3c 0367ff58 fbserver!
> ISC_event_wait+0x91 (FPO: [EBP 0x0367ff50] [6,16,0]) (CONV: cdecl)
> 0367ff50 00418a3e 00000001 00000000 00404b39 fbserver!
> cache_writer+0x249 (CONV: stdcall)
> 0367ff5c 00404b39 00000000 00614fb8 00614ef0 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 0367ff84 7c349565 0091a670 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 0367ffb8 77e6608b 00614ef0 00000000 00000000 MSVCR71!endthreadex+0xa0
> 0367ffec 00000000 7c3494f6 00614ef0 00000000 kernel32!
> GetModuleFileNameA+0xeb



> *----> State Dump for Thread Id 0x708 <----*

This is garbage_collector thread. It is sleep now

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 038bfd54 77e6109d 00000001 038bfd98 00000001 ntdll!KiFastSystemCallRet
> 038bfd70 00403ac1 00000001 038bfd98 00000001 kernel32!
> WaitForMultipleObjects+0x18
> 038bfdd4 00476c8a 00000001 038bff2c 038bff24 fbserver!
> ISC_event_wait+0x91 (FPO: [EBP 0x038bff50] [6,16,0]) (CONV: cdecl)
> 038bff50 00418a3e 0042e630 00000000 00404b39 fbserver!
> garbage_collector+0x68a (CONV: stdcall)
> 038bff5c 00404b39 00000000 00614a38 00614ef0 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 038bff84 7c349565 0091a648 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 038bffb8 77e6608b 00614ef0 00000000 00000000 MSVCR71!endthreadex+0xa0
> 038bffec 00000000 7c3494f6 00614ef0 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0x4e8 <----*

This is another garbage_collector thread (we have it per database too).
It is sleep also

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 03abfd54 77e6109d 00000001 03abfd98 00000001 ntdll!KiFastSystemCallRet
> 03abfd70 00403ac1 00000001 03abfd98 00000001 kernel32!
> WaitForMultipleObjects+0x18
> 03abfdd4 00476c8a 00000001 03abff2c 03abff24 fbserver!
> ISC_event_wait+0x91 (FPO: [EBP 0x03abff50] [6,16,0]) (CONV: cdecl)
> 03abff50 00418a3e 0003e630 00000000 00404b39 fbserver!
> garbage_collector+0x68a (CONV: stdcall)
> 03abff5c 00404b39 00000000 00614e28 00614ef0 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 03abff84 7c349565 009151b4 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 03abffb8 77e6608b 00614ef0 00000000 00000000 MSVCR71!endthreadex+0xa0
> 03abffec 00000000 7c3494f6 00614ef0 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0xabc <----*

This is worker thread. It is sleep now as it is not scheduled to run currently.

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 047eff14 77e6ba42 0000006c 0000ea60 00000000 ntdll!KiFastSystemCallRet
> 047eff28 00564509 0000006c 0000ea60 03e04b1c kernel32!
> WaitForSingleObject+0x12
> 047eff50 00418a3e 0401723c 00000000 00404b39 fbserver!
> loopThread+0x1c9 (FPO: [EBP 0x047eff84] [1,3,0]) (CONV: stdcall)
> 047eff5c 00404b39 00000000 00614ef0 00615310 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 047eff84 7c349565 03e04b1c 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 047effb8 77e6608b 00615310 00000000 00000000 MSVCR71!endthreadex+0xa0
> 047effec 00000000 7c3494f6 00615310 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0x39c <----*

Again stalled worker thread

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 00f3ff14 77e6ba42 0000006c 0000ea60 00000000 ntdll!KiFastSystemCallRet
> 00f3ff28 00564509 0000006c 0000ea60 04020014 kernel32!
> WaitForSingleObject+0x12
> 00f3ff50 00418a3e 0401723c 00000000 00404b39 fbserver!
> loopThread+0x1c9 (FPO: [EBP 0x00f3ff84] [1,3,0]) (CONV: stdcall)
> 00f3ff5c 00404b39 00000000 00615080 00615310 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 00f3ff84 7c349565 04020014 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 00f3ffb8 77e6608b 00615310 00000000 00000000 MSVCR71!endthreadex+0xa0
> 00f3ffec 00000000 7c3494f6 00615310 00000000 kernel32!
> GetModuleFileNameA+0xeb


> *----> State Dump for Thread Id 0xd8 <----*

Again stalled worker thread

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 04a2ff14 77e6ba42 0000006c 0000ea60 00000000 ntdll!KiFastSystemCallRet
> 04a2ff28 00564509 0000006c 0000ea60 03e04cdc kernel32!
> WaitForSingleObject+0x12
> 04a2ff50 00418a3e 00000000 00000000 00404b39 fbserver!
> loopThread+0x1c9 (FPO: [EBP 0x04a2ff84] [1,3,0]) (CONV: stdcall)
> 04a2ff5c 00404b39 00000000 006151c8 00615310 fbserver!
> ThreadPriorityScheduler::run+0xe (FPO: [0,0,0]) (CONV: thiscall)
> 04a2ff84 7c349565 03e04cdc 00000000 00000000 fbserver!`anonymous
> namespace'::threadStart+0x49 (CONV: stdcall)
> 04a2ffb8 77e6608b 00615310 00000000 00000000 MSVCR71!endthreadex+0xa0
> 04a2ffec 00000000 7c3494f6 00615310 00000000 kernel32!
> GetModuleFileNameA+0xeb



> *----> State Dump for Thread Id 0xefc <----*

This is worker thread which is doing some job.

> function: fbserver!MET_format
> 00434d87 7504 jnz fbserver!MET_format+0x2d
> (00434d8d)
> 00434d89 33c0 xor eax,eax
> 00434d8b eb08 jmp fbserver!MET_format+0x35
> (00434d95)
> 00434d8d 8b400c mov eax,[eax+0xc]
> 00434d90 2bc1 sub eax,ecx
> 00434d92 c1f802 sar eax,0x2
> 00434d95 0fb7542434 movzx edx,word ptr [esp+0x34]
> 00434d9a 3bd0 cmp edx,eax
> 00434d9c 730b jnb fbserver!MET_format+0x49
> (00434da9)
> 00434d9e 8b0491 mov eax,[ecx+edx*4]
> 00434da1 85c0 test eax,eax
> 00434da3 0f85f9010000 jne fbserver!MET_format+0x242
> (00434fa2)
> 00434da9 56 push esi
> 00434daa 57 push edi
> 00434dab 6a01 push 0x1
> 00434dad 6a08 push 0x8
> 00434daf 53 push ebx
> 00434db0 33f6 xor esi,esi
> 00434db2 e839620100 call fbserver!CMP_find_request
> (0044aff0)
> 00434db7 8bf8 mov edi,eax
> 00434db9 83c40c add esp,0xc
>
> *----> Stack Back Trace <----*
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\WINDOWS\system32\USER32.dll -
> ChildEBP RetAddr Args to Child
> 04e2f540 00472d01 04e2fba4 033ff1b8 00000001 fbserver!MET_format+0x41
> (FPO: [EBP 0x00000000] [3,8,0]) (CONV: cdecl)
> 04e2f560 00473107 04e2fba4 040510ac 00000000 fbserver!VIO_record+0x31
> (FPO: [EBP 0x040510ac] [4,0,0]) (CONV: cdecl)
> 04e2f9a0 00475851 04e2fba4 040510ac 04050088 fbserver!VIO_data+0x47
> (FPO: [EBP 0x04050edc] [3,263,0]) (CONV: cdecl)
> 04e2f9bc 004cfe84 04e2fba4 040510ac 040e2e4c fbserver!VIO_get+0x61
> (FPO: [5,0,0]) (CONV: cdecl)
> 04e2f9f0 004d01db 040ae6f0 040e2e4c 040e2d0c fbserver!
> get_record+0x104 (FPO: [EBP 0x04050edc] [4,3,0]) (CONV: cdecl)
> 04e2fa20 004cfa81 04e2fb00 040e2d0c 00000000 fbserver!
> get_record+0x45b (FPO: [EBP 0x03db25d0] [4,3,0]) (CONV: cdecl)
> 04e2fa4c 004d0386 04e2fba4 03db25d0 00000004 fbserver!
> fetch_record+0x71 (FPO: [EBP 0x04050edc] [3,0,0]) (CONV: cdecl)
> 04e2fa74 7738a5d4 04051474 00000026 004d08d8 fbserver!
> get_record+0x606 (FPO: [EBP 0x04050edc] [4,3,0]) (CONV: cdecl)
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> 04e2faac 0046b385 04e2fba4 03db25d0 00000000 USER32!
> CharUpperBuffA+0x86
> 0000000a 00000000 00000000 00000000 00000000 fbserver!looper+0x345
> (CONV: cdecl)



> *----> State Dump for Thread Id 0xbac <----*

I don't know what is it. Perhaps it is related with how drWatson killed precess

> *----> Stack Back Trace <----*
> ChildEBP RetAddr Args to Child
> 0421fff4 00000000 00000000 00000000 00000000 ntdll!DbgBreakPoint


So, what we can see here ? Superserver running as service. One user database
is attached (second is security database). One request are active.

From this info i can say that you must carefully check PLAN for that statement
which was running at time you killed the server.

Regards,
Vlad.