| Subject | operating system directive CreateFile failed on security database under load |
|---|---|
| Author | Moebius, Dr. Jonathan |
| Post date | 2018-10-01T15:10:05Z |
Dear all,
in our jenkins-server, when testing our application, which is based on firebird, we are seeing errors of the type "operating system directive CreateFile failed". Recently, we have seen additional failures "could not attach to security database", which can be tracked to the server log, where the cause is again "CreateFile failed" with the OS-message "zugriff verweigert" (access denied). These failures are sporadic. We have not yet a solid pattern when these errors occur, however, the issue seems to be more likely to occur, when a lot of load is on the firebird server.
To understand the scenario: We are using firebird/jaybird for the automated tests within our Java-based-application. For our tests, we have a variety of roughly 30 databases which represent specific configurations of our application. As the testbase is quite large, it is split into multiple testsets (which may access all 30 of the testdatabases). There is a central storage for these databases and prior to a test-execution, copies of the databases are made into the jenkins-workspace. The test itself will only access those copies, however multiple copies of the same database (with the same name) may be in use simultaniously. A jenkins node in our case will execute at most three testsuites at a time and will have its own Firebird-Server instance.
We are using Firebird-Superserver in 2.5.5, the operating System is Windows Server 2012 und we ware accessing via jaybird 2.2.10. Firebird is running in its default-configuration. We are aware, that this may be problematic and are currently trying to establish a suitable configuration for our needs.. We are rather free with respect to experiments within the firebird configuration.
The issue did not occur suddenly but has been slowly becoming more frequently until it has become a problem. We have tried moving some of our jenkins nodes to new hardware, which is SSD based (instead of the HDD-Storage used until now). This experiment is only a week old, up to now, no such errors have been recorded on the new machines. This strengthens our suspicion, that the root cause is some form of I/O-overload on the jenkins-nodes. However, it could be something else entirely, for example simply concurrent access to the security-database.
So, any help will be appreciated. The four questions, we have come up for now are:
Has anyone an idea, how to analyse this problem more thouroughly?
Is it legal to use multiple gdb-databases with the same filename (although different paths) within the same firebird instance?
Is it possible, that temporary files created by firebird cause a name-clash?
Is SuperServer a suitable architecture for our scenario with several client processes or would SuperClassic be better suited?
Kind regards,
Jonathan
in our jenkins-server, when testing our application, which is based on firebird, we are seeing errors of the type "operating system directive CreateFile failed". Recently, we have seen additional failures "could not attach to security database", which can be tracked to the server log, where the cause is again "CreateFile failed" with the OS-message "zugriff verweigert" (access denied). These failures are sporadic. We have not yet a solid pattern when these errors occur, however, the issue seems to be more likely to occur, when a lot of load is on the firebird server.
To understand the scenario: We are using firebird/jaybird for the automated tests within our Java-based-application. For our tests, we have a variety of roughly 30 databases which represent specific configurations of our application. As the testbase is quite large, it is split into multiple testsets (which may access all 30 of the testdatabases). There is a central storage for these databases and prior to a test-execution, copies of the databases are made into the jenkins-workspace. The test itself will only access those copies, however multiple copies of the same database (with the same name) may be in use simultaniously. A jenkins node in our case will execute at most three testsuites at a time and will have its own Firebird-Server instance.
We are using Firebird-Superserver in 2.5.5, the operating System is Windows Server 2012 und we ware accessing via jaybird 2.2.10. Firebird is running in its default-configuration. We are aware, that this may be problematic and are currently trying to establish a suitable configuration for our needs.. We are rather free with respect to experiments within the firebird configuration.
The issue did not occur suddenly but has been slowly becoming more frequently until it has become a problem. We have tried moving some of our jenkins nodes to new hardware, which is SSD based (instead of the HDD-Storage used until now). This experiment is only a week old, up to now, no such errors have been recorded on the new machines. This strengthens our suspicion, that the root cause is some form of I/O-overload on the jenkins-nodes. However, it could be something else entirely, for example simply concurrent access to the security-database.
So, any help will be appreciated. The four questions, we have come up for now are:
Has anyone an idea, how to analyse this problem more thouroughly?
Is it legal to use multiple gdb-databases with the same filename (although different paths) within the same firebird instance?
Is it possible, that temporary files created by firebird cause a name-clash?
Is SuperServer a suitable architecture for our scenario with several client processes or would SuperClassic be better suited?
Kind regards,
Jonathan