On 1-10-2018 17:10, 'Moebius, Dr. Jonathan' jonathan.moebius@...
[Firebird-Java] wrote:

> in our jenkins-server, when testing our application, which is based on
> firebird, we are seeing errors of the type "operating system directive
> CreateFile failed". Recently, we have seen additional failures "could
> not attach to security database", which can be tracked to the server
> log, where the cause is again "CreateFile failed" with the OS-message
> "zugriff verweigert" (access denied). These failures are sporadic. We
> have not yet a solid pattern when these errors occur, however, the issue
> seems to be more likely to occur, when a lot of load is on the firebird
> server.
>
> To understand the scenario: We are using firebird/jaybird for the
> automated tests within our Java-based-application. For our tests, we
> have a variety of roughly 30 databases which represent specific
> configurations of our application. As the testbase is quite large, it is
> split into multiple testsets (which may access all 30 of the
> testdatabases). There is a central storage for these databases and prior
> to a test-execution, copies of the databases are made into the
> jenkins-workspace. The test itself will only access those copies,
> however multiple copies of the same database (with the same name) may be
> in use simultaniously. A jenkins node in our case will execute at most
> three testsuites at a time and will have its own Firebird-Server instance.

It would be good to know exactly how those copies are made. It sounds as
if the copy process may not have released its file locks before Firebird
tries to open the database file.

Also, check if the Firebird database files (including the security
database) are excluded from your virus scanner. It is possible that a
virus scanner has a shared lock on the file because it is scanning it as
the file was just created/copied/modified. If there is a slight pause
between tests, this may even apply to the security database as Firebird
might release its connection to that file as well.

As Firebird SuperServer requires exclusive access, this may result in an
access denied if another process (the copy process or the virus scan)
still has a lock on that file.

> We are using Firebird-Superserver in 2.5.5, the operating System is
> Windows Server 2012 und we ware accessing via jaybird 2.2.10. Firebird
> is running in its default-configuration. We are aware, that this may be
> problematic and are currently trying to establish a suitable
> configuration for our needs.. We are rather free with respect to
> experiments within the firebird configuration.

Consider updating to Firebird 2.5.8; looking at the release notes I
don't really see something similar (except maybe CORE-5067), but it may
help.

For testing, I don't think there will be any problems with running
Firebird using its default configuration.

> The issue did not occur suddenly but has been slowly becoming more
> frequently until it has become a problem. We have tried moving some of
> our jenkins nodes to new hardware, which is SSD based (instead of the
> HDD-Storage used until now). This experiment is only a week old, up to
> now, no such errors have been recorded on the new machines. This
> strengthens our suspicion, that the root cause is some form of
> I/O-overload on the jenkins-nodes. However, it could be something else
> entirely, for example simply concurrent access to the security-database.

That reinforces my theory that this could be a virus scanner. More I/O
could mean that the virus scan takes longer (lower priority process),
switching to an SSD could mean the scan is quicker and completes before
your test tries to open the file.

> So, any help will be appreciated. The four questions, we have come up
> for now are:
>
> Has anyone an idea, how to analyse this problem more thouroughly?

You could try to trace things with Sysinternals process monitor to see
which files get opened and closed by which process (this can be pretty
overwhelming if you don't setup the filters right though, consider
filtering on your database files and the security database file). This
might help in identifying the cause of those access denied errors.

That said, my experience with troubleshooting these types of problems is
limited. You may want to consider posting a follow up question on
firebird-support as well.

> Is it legal to use multiple gdb-databases with the same filename
> (although different paths) within the same firebird instance?

That shouldn't be a problem, as Firebird uses the full path, not just
the filename.

One thing to consider is to rename your Firebird database. The .gdb
extension is an historic extension derived from one of Firebird
ancestors, these days .fdb is considered 'standard'.

I have heard stories that using the extension .gdb may be problematic on
Windows. However beyond vague allusions in a Firebird 1.0 quickstart
document, I'm not sure of the details, so this may be apocryphal.

> Is it possible, that temporary files created by firebird cause a
> name-clash?

It shouldn't.

> Is SuperServer a suitable architecture for our scenario with several
> client processes or would SuperClassic be better suited?

I don't think it will make a difference, but you could always try. If
you are using (custom) UDFs, it might make sense to use Classic (not
SuperClassic), as in that case, things that might actually crash the
server (eg bugs in UDFs, or in Firebird itself for that matter), only
affect a single connection (usually).

Mark
--
Mark Rotteveel