Subject Re: [firebird-support] connection refused, fbserver processes rising
Author Bogusław Brandys
mb22765 wrote:
> Hello all,
>
> last week we experienced some strange problems with our firebird
> server. Any connections were refused while the number of fbserver
> processes is rising. Details:
>
> - FirebirdSS-1.5.2.4731-0 running on SuSE Standard Server 8 with
> kernel k_smp-2.4.21-273
> - Clients are various selfmade applications written in Borland Delphi
>
> Wednesday morning we got some error reports that the DB refuses any
> connections. We found that there were more than 1000 fbserver
> processes running (usually < 20). Stopping firebird, killing all
> processes and restarting made everything work again. In the
> firebird.log we found a number of "INET/inet_error: read errno = 104".
> We never had such trouble in the last 2 years, so we just thought:
> Sometimes shit happens, kept an eye on the number of fbserver
> processes, but everything was fine for the next few days.
>
> On saturday I had to shut down the server for some maintenance,
> unrelated to fb. The shutdown process hung several minutes on shutting
> down the firebird server. SSH was still working, so I could log on and
> found the usual number of fbserver processes. When I killed one of
> them the shutdown process continued. Nothing in the log - only one
> line "bin/fbserver terminated abnormally" because of my "kill" command.
>
> After finishing maintenance the server was restarted. I checked if fb
> was working with one of our Delphi DB client applications. Everything
> OK. Half an hour later when I had left, our network monitoring tool
> nagios (see www.nagios.org) threw some messages like "Firebird Service
> Check Timed Out". I could see in the system statistics of the SuSE
> StdS8 that the number of processes increased from this time on to
> about 1000 within 14 hours. Monday morning we were in trouble again.
> No connection possible. So again I did "rcfirebird stop" and "killall
> fbserver". Here's the part of firebird.log from startup on saturday to
> "killall" on monday morning:
>
> samba1 (Client) Sat Mar 18 17:31:48 2006
> /opt/firebird/bin/fbguard: guardian starting bin/fbserver
> samba1 (Client) Mon Mar 20 07:59:54 2006
> INET/inet_error: read errno = 104
> samba1 (Client) Mon Mar 20 07:59:54 2006
> INET/inet_error: receive in try_connect errno = 104
> samba1 (Client) Mon Mar 20 08:00:53 2006
> /opt/firebird/bin/fbguard: bin/fbserver terminated abnormally (-1)
>
> Nothing in /var/log/messages or /var/log/warn.
>
> We have some processes on different clients that try to connect to the
> database every few minutes: nagios and two of our own Delphi apps.
> Something seems to go wrong in a way that every db connection gets
> refused and each connection attempt leaves an fbserver process in memory.
>
> My questions:
> - What does the error code 104 in the log mean? I also find some
> errno=4 and 9 in the logs. On firebird.sourceforge.net I could only
> find the SQL error table. Where's the documentation for the error
> codes in the logfile?
>
> - Does anybody have an idea what could cause this behaviour? And how
> to fix it?
>
> Thank you,
>
> Mario Balfanz
> FAFNIR GmbH
> Hamburg, Germany
>

Simple advice : first - find what was changed during last days (when
problems started to arise).


Regards
Boguslaw