Subject | Huge Firebird memory usage using FDB |
---|---|
Author | josef_gschwendtner |
Post date | 2013-04-30T19:20:56Z |
Hi,
We got a problem when we use the Firebird driver for Python (fdb) inside a connection pool. If a single connection is used for the execution of
many queries and each query uses a separate cursor instance we observed an increasing memory usage on the server side (the fbserver process).
Only when the connection gets closed the memory usage of the fbserver process gets back to a "normal" value.
An equal situation can be simulated with this short python script.
import fdb
con = fdb.connect(dsn='localhost/3050:C:\\Development\\DB\\mydb.fdb', user='sysdba',
password='masterkey')
for i in range(10000):
cur = con.cursor()
cur.execute( "select * from RDB$Relations")
for row in cur.fetchall():
print i, row[0]
con.commit()
t = raw_input('Press Key')
con.close()
print "Finished"
When the loop has finished and before closing the connection the fbserver process uses almost one GB of memory. This is independent of the table being used.
RDB$Relations is only used for demonstration. The same happens also with every other table. After closing the connection everything is fine on the server
(fbserver memory usage is back to 7 MB).
We found out that releasing the Cursor instance does not permanently free the prepared statement on the server side. During the destructor of the Cursor instance
the internal managed PreparedStatement instances will also be released. This will call the method _close from PreparedStatement. Inside the _close Method the
release of the server side statement resource will only be triggered if there is an open connection. But to get this connection the PreparedStatement instance uses the
cursor Attribute which is a weak reference to the Cursor instance. But this Cursor instance is no longer available because the program execution is inside the
destructor (__del__) of this cursor. The weak reference is dead.
It seems that Cursor and PreparedStatement instances are released on the client side but the following statement from the _close method of PreparedStatement doesn't get executed.
api.isc_dsql_free_statement(self._isc_status, stmt_handle,
ibase.DSQL_drop)
But this statement is required for releasing the server side resources and also the memory used for these resources.
Thanks for your help,
Josef