Summary:
All clients freeze until FBServer is restarted




Background:
For the past 6 months plus, a couple of our customers have been
experiencing our client application locking up for all users. We have
spent considerable resources trying to understand the issue but have
made little progress and are now reaching out to the Firebird community
for suggestions on how to overcome or at least minimize occurrence of
this issue




Typical database installation:
* SuperServer 1.5.6.5026 SuperServer running as a service with FBGuardian
* Borland C++ 2007 VCL with IB Objects (8/25/2007 Sub-Release 4.8 Build
7) based client applications
* Client application is basically single threaded event driven
* Single FB database on server only
* Typically a dedicated MS-Windows based machine for the database server
* Each client has two connections to DB that are open for the life time
of the app (1 for data, 1 for events)
* Typically sub 3GB fdb file
* Typically 3-20 clients per database installation






70+ installations across the globe work fine but 2 are suffering:
* all connected clients freezing concurrently (within a minute of each
other)
* sometimes 3 or more times a day
* sometimes can go 5-10 days without an issue then multiple freezes
* clients remain locked up until FBServer is restarted at which point
they suddenly spring into life reporting the connection is lost
* Worse case the freeze can happen 10 mins or so after restarting the
FBServer
* restarting the server machine does not help the issue i.e. just as
likely to happen as just restarting FBServer via Control Panel
* FBServer at freeze point does not show high CPU or memory consumption
in task manager
* Waiting 10 mins and still the clients do not recover
* Nothing unusual in MS-Windows event table
* Nothing unusual in Firebird logs
* No Anti-virus running on servers
* No firewall running on servers
* Full firewall exceptions given for client applications
* Bizarrely seems to be far worse on Thursday and Fridays for one of the
installations (no crash pattern yet noted at the other installation)
* other client applications connected to the same database and using
same build technology but with different exe names do not appear to be
impacted (though they are far less intensive and so this may be
misinformation)
* Client application running directly on the server also freezes
* Both these installations are running MS-Windows 7 as their servers
(but so are working installations)
* Seems to be event related, as a typical madExcept call stack at the
point of freeze shows 'isc_que_events':


778af8ca +00e ntdll.dll NtWaitForSingleObject
75472f7b +05b ws2_32.dll WahReferenceContextByHandle
75476a25 +09c ws2_32.dll select
064b6d59 +069 fbclient.dll isc_que_events
0064e92e +06e nCall.exe Ib_components TIB_Component.CheckSession
00773157 +06f nCall.exe Ib_events TIB_Events.API_QueueEvents
007729d1 +09d nCall.exe Ib_events TIB_Events.RegisterEvents
0040ebf4 +7ec nCall.exe TSubscriber.cpp 287 +68 TSubscriberIBO.SetEvents
0040d40b +2eb nCall.exe TSubscriber.cpp 67 +9 TSubscriber.AddEvent
004bf350 +fa8 nCall.exe TdmMain.cpp 400 +170 TdmMain.Connect




We have tried:
* Backup and restore has no benefit with freezes experienced within a
few hours of a backup and restore
* Daily restart of the database server - no effect
* The freeze is not related to performing a backup
* The freeze is not related to performing an intense job
* Clients on static and dynamic IPs - no effect
* Firewall exceptions have been verified at both installations with
program exceptions for the clients
* RemoteAuxPort values of 0, 3051, 40000 tried with no noticeable benefit
* We've tried registering for more events
* We've tried ensuring an event is generated every minute (in an attempt
to keep the event port open in case it is being closed through lack of use)
* 3rd party network experts tell us there are no networking working
issues being experienced by other applications
* Users have suggested that the client application normally slows down
before the freezing (not verified)
* FBClient.dlls 1.52 and 1.56 (including ensuring all clients are using
same version)




Having read many postings regarding similar issues, only these two
appear useful but don't indicate any real solutions or if the issue has
been resolved in a later version of Firebird.
*
https://groups.yahoo.com/neo/groups/firebird-support/conversations/messages/75364
*
https://groups.yahoo.com/neo/groups/firebird-support/conversations/messages/75366




Suggestions on what to try next sought!
Paul