I have been chasing a problem with our webservers. We have a
custom written ISAPI dll which has been running fine for many years
(though it is constantly upgraded with new features).
Over the past 6 weeks we have been having firebird fail on both
servers nearly simultaneously. It was happening once a week, then
for a span it was several times a day, now we went 12 days and
thought we were out of the woods and it happened again this morning.
We are running FB 1.5.5 on Windows 2003 server (fully updated).
The isapi dll will run fine, processing thousands of requests (it
does about 1.5 million page requests per day). At some point during
the day it will receive this error:

Unable to complete network request to host "localhost".
Failed to establish a connection.
Access is denied.

I was originally running 1.5.0 when this started happening. I
upgraded to 1.5.5 and things actually got a bit worse. With 1.5.0
the error message received was

Unable to complete network request to host "localhost".
Error writing data to the connection.
An existing connection was forcibly closed by the remote host.

When the ISAPI dll sees an error like this it is programmed to
restart IIS. Under 1.5.0, that was all it took, things were back to
working.
After upgrading to 1.5.5 (and the error changing to "Failed to
establish a connection") the only way to fix the problem is to reboot
the server.
In my ISAPI error report, I have it list the running processes, etc
at the time of the error. I noticed that when it crashed, the last
created entry in the process list was CMD.EXE.
I turned on server security logging for application start / exit
and it turns out that just before the crash I see this entry in the
security log:

Event Type: Success Audit
Event ID: 592
Date: 3/20/2008
Time: 5:37:50 AM
A new process has been created:
New Process ID: 3604
Image File Name: C:\WINDOWS\system32\cmd.exe
Creator Process ID: 1684


the "creator process id" 1684 is

Event Type: Success Audit
Event ID: 592
Date: 3/20/2008
Time: 1:01:48 AM
Description:
A new process has been created:
New Process ID: 1684
Image File Name: C:\Program Files\Firebird\Firebird_1_5
\bin\fbserver.exe



There is nothing in the firebird.log, we do a 1 am reboot to try to
help the situation:

ROYSTON (Client) Thu Mar 20 01:01:48 2008
Guardian starting: C:\Program Files\Firebird\Firebird_1_5
\bin\fbserver.exe


ROYSTON (Client) Thu Mar 20 05:40:43 2008
Guardian starting: C:\Program Files\Firebird\Firebird_1_5
\bin\fbserver.exe


Since both servers are failing within moments of each other, it
seems like some external source is attacking our server. We did one
one of these attacks have a virus successfully attack our server, but
it was quickly cleaned out.

So, what can cause FBServer.EXE to start a CMD.EXE ? And more
importantly, what steps can I take to make this stop. Especially now
that it causes our web server to have to be rebooted.
Any help would be appreciated.