Subject Re: Fwd: Invalid connection after JBoss IdleRemover-Action (NPE)
Author Simon Ee
Hi,

Thought I'd share our experience that looks very, very similar. We
worked around the issue by setting
<idle-timeout-minutes>0</idle-timeout-minutes>
on each datasource. Haven't had a problem since. (It's been about 10
months).

Additional observations on our situation, prior to applying the workaround:

It appeared as if on the Jaybird side, connections were being released
after an idle timeout, but on the FB side, the number of active
connections in use grew steadily. (All our customer sites run JBoss on
Linux with Firebird CS, so it was easy to see this.) Sometimes, it
appeared that connections were released properly, but the number of
fb_inet_server processes never really subsided to the minimum. The
trend was always on the upswing, even with a constant number of users (<
10) and stable workload during the day, and of course zero activity
(except backup) at night.

Additionally, Checking netstat shows some fb_inet_server processes with
non-zero SendQ, suggesting that the other end of the connection was
somehow not responding, but still maintaining the connection. Quite
often, on the morning after a particularly heavy day, users called in to
compain that they could not even log in to the app. (The user database
is also in FB).

We didn't report the error because our first instinct was to audit our
own code, and also to check customers' server config/environment against
our own. We eventually manage to replicate the problem on our own by
simulating 5 active users. The above symptoms usually manifest within a
day or two.

Hope this helps, and sorry it didn't come out earlier for everyone's
benefit. At least, I can confidently say that disabling idle timeout
fixes this problem, as we haven't had a reccurance at any of our
customers' production systems for the last 10 months.

Simon.


Firebird-Java@yahoogroups.com wrote:

>________________________________________________________________________
>________________________________________________________________________
>
>Message: 16
> Date: Mon, 20 Dec 2004 22:47:36 +0100
> From: "Roman Rokytskyy" <rrokytskyy@...>
>Subject: Fwd: Invalid connection after JBoss IdleRemover-Action (NPE)
>
>Hi Josef,
>
>Below is reply from David Jencks. It seems that we're in agreement how to
>resolve the issue, but the changes are not small and require thorough
>testing. This does not fit the point release. I will try to fix it in
>JayBird 1.6.
>
>The only workaround that comes to my mind it to disable idle remover in
>JBoss (in theory that should happen when min and max connections values are
>the same).
>
>Roman
>
>==========================================================================
>* Forwarded by Roman Rokytskyy <rrokytskyy@...>
>* From: "David Jencks" <david@...>
>* Date: Mon, 20 Dec 2004 11:45:39 -0800
>* To: "Roman Rokytskyy" <rrokytskyy@...>
>* Subj: Re: [Firebird-Java] Invalid connection after JBoss
>IdleRemover-Action (NPE)
>==========================================================================
>
>hmmm... I'm wondering why the managed connection doesn't look for a new
>physical connection if its previous physical connection has been
>closed? I think the situation must be that two managed connections are
>sharing a physical connection, and one of the managed connections is
>destroyed, also closing the physical connection. Perhaps we are not
>propagating the fact that the physical connection was closed back to
>the still-open managed connection?
>
>I think the best solution will be to remove the code that shares
>physical connections among managed connections.
>
>I doubt this is directly influenced by the track-connection-by-tx
>setting, it is probably a problem with either setting.
>
>thanks
>david jencks
>
>On Dec 20, 2004, at 4:47 AM, Roman Rokytskyy wrote:
>
>
>
>>Hi David,
>>
>>Can this be related to the issue with "track-connection-by-tx" setting
>>in
>>JBoss DS config?
>>
>>Thanks!
>>Roman
>>
>>
>>
>>>Hi,
>>>
>>>
>>>one of our customers is getting sporadically NullPointerExceptions
>>>(which last for a certain period).
>>>
>>>
>>>To avoid the great number of NPE's we have changed methode
>>>isc_start_transaction in Jaybird to log a message on db.out == null
>>>and throw an exception afterwards. This has improved the situation
>>>because instead of many NPE's we get one exception and afterwards
>>>the program works as expected, again.
>>>
>>>
>>>One of our customers had this fault-situation (db.out == null) 5
>>>times last week. We found out, that the situation always occures
>>>immediately after the JBoss IdleRemover-thread was active and
>>>probably has freed a connection which was used by another thread
>>>(also the situation occures not each time the IdleRemover-thread was
>>>active). We think there might be a synchronisation problem.
>>>
>>>
>>>Another customer runs the same program without any problems.
>>>Both custumers use RedHat 7.2.
>>>- Customer with problems uses kernel 2.4.20-28 (the newer one)
>>>- Customer without problems uses kernel 2.4.7-108
>>>
>>>
>>>Other Conditions:
>>>- Software: Java 1.4.2_03, JBoss 3.2.3, Jaybird 1.5.5
>>>- Hardware: Dual Xeon CPU (3,06 GHz), 2 GB RAM
>>>
>>>
>>>Any help would be greatly appreciated.
>>>Josef Gschwendtner
>>>
>>>
>>
>>
>>>Yahoo! Groups Links
>>>
>>>
>>
>>
>
>==========================================================================
>With best regards, Roman Rokytskyy. E-mail: rrokytskyy@...
>
>
>
>