Subject Re: fast growing database
Author miekejanson
Goodmorning,

Now the next thing happened.....

All users worked the database yesterday. I made a backup at 4:30,
everything
fine. They close my application (like they do the last 7 years).
Then I connect using my RDP-connection, and try to make a backup using
the dbworkbench. At that moment nobody is connected. This fails!


I tried several things like deactivating (actually removing) indexes,
triggers and foreign keys. Backup only metadata and then pump the data
into the empty database. but, it did not work... the pump stops in
the same table, (at different records)
I know there is one table corrupt, because if I want to open it ( I
use the dbworkbench) and query the last inserted records, the
dbworkbench hangs.

How can I find out what is going wrong? I'm convinced something goes
wrong when the last user disconnects....


The stats yesterday:

Database header page information:
Flags 0
Checksum 12345
Generation 134832
Page size 8192
ODS version 11.0
Oldest transaction 45907
Oldest active 134823
Oldest snapshot 134823
Next transaction 134824
Bumped transaction 1
Sequence number 0
Next attachment ID 0
Implementation ID 16
Shadow count 0
Page buffers 1000
Next header page 0
Database dialect 1
Creation date Sep 2, 2008 21:09:43
Attributes force write

Variable header data:
Sweep interval: 20000
*END*

And the stats from the day before:
Database "C:\VVVWijzerDataNew\VVV.fdb"
Database header page information:
Flags 0
Checksum 12345
Generation 71450
Page size 8192
ODS version 11.0
Oldest transaction 45907
Oldest active 67376
Oldest snapshot 63944
Next transaction 71442
Bumped transaction 1
Sequence number 0
Next attachment ID 0
Implementation ID 16
Shadow count 0
Page buffers 1000
Next header page 0
Database dialect 1
Creation date Sep 2, 2008 21:09:43
Attributes force write

Variable header data:
Sweep interval: 20000
*END*


Are the differences alarming to you?
The first day, the database grew with 20 Mb, and the next day with
about 10 Mb.....


About the XNet -error:
Does firebird 1.5 works different from FB2.0 at that point? I have
been using the same connectionstring for years without problems..
Also, it seems to me, that is the connectionstring is wrong, the
connection will NEVER be established, and not every now and than...
but of course, we will try...


Thank you for your time....
Mieke


--- In firebird-support@yahoogroups.com, Helen Borrie <helebor@...> wrote:
>
> At 21:52 4/09/2008, you wrote:
> >Good morning,
> >
> >We have problems with a firebird2.0 database after we recently
> >upgraded from
> >fb1.5 to fb2.0.
> >Our customers work the whole day with the database. Normally the
database
> >grows about 2 MB per day, now it is 20MB.
>
> You have just restored this database. A freshly restored database
has no recycled space waiting to be filled up so, for a time, you will
see "growth" as the engine acquires enough space to accommodate that
high level of garbage that you are building up.
>
> >A normal backup/restore does not work.
>
> Explain what "does not work" means.
>
>
> >Here you see the header information. As you can see there is a big gap
> >between the oldest transaction and the next transaction.
>
> You need to figure out whether that gap is big, considering what you
are doing. Is this the first time you have ever monitored the gap?
>
> >How can that occur,
>
> A gap occurs all the time, really. As long as a transaction remains
"interesting" the old record versions that it has generated will not
be garbage collected...
>
> >do we need to fix it,
>
> If the garbage buildup is causing problems, you fix it by fixing
your application code so that idle transactions get committed. If you
have not changed the application code then this is not a new problem -
it has always been there.
>
> If you are using Superserver and you also used SS previously, then
it is possible that the effect of the garbage buildup is being
heightened due to the different way that the Fb 2 engine manages
garbage. By default it uses both background GC and cooperative GC.
You can revert the GC policy (GCPolicy in firebird.conf) to
background alone - I leave it to you to read the release notes
regarding this. It won't fix any intrinsic faults in your transaction
management but it will get you back to a similar level to Fb 1.5,
which does only background GC.
>
> If you changed from SS to Classic then you have only cooperative GC.
>
> >can we fix it while the database is connected?
>
> You can run a sweep while the database is connected. You can edit a
config setting while the database is connected, too, but it won't take
effect until the *SERVER* is shut down and restarted.
>
>
> >Database header page information:
> > Flags 0
> > Checksum 12345
> > Generation 71450
> > Page size 8192
> > ODS version 11.0
> > Oldest transaction 45907
> > Oldest active 67376
> > Oldest snapshot 63944
> > Next transaction 71442
> > Bumped transaction 1
> > Sequence number 0
> > Next attachment ID 0
> > Implementation ID 16
> > Shadow count 0
> > Page buffers 1000
> > Next header page 0
> > Database dialect 1
> > Creation date Sep 2, 2008 21:09:43
> > Attributes force write
> >
> > Variable header data:
> > Sweep interval: 20000
> > *END*
>
> According to these stats you have started about 71.5K transactions
in less than two days. Transaction number 45907 is still interesting
but it is not active. That means you have an unknown number of
younger transactions whose garbage is "stuck". Your oldest active
transaction is quite recent, compared to your throughput, i.e. the
gaps between the oldest active and oldest snapshot, and OS and the
next transaction, respectively, are relatively small. Your database
is holding garbage, though. But these numbers are not
excessive...keep monitoring to see if there is movement.
>
>
> >The next problem (no idea if this has anything to do with the above):
>
> It doesn't. ;-)
>
>
> >If I try to startup my application with the Remote desktop
connection this
> >fails, and in the firebird logfile I
> >see the following message:
> > XNET error (xnet:2043) connection lost: another side is dead
> >Does this message mean anything to you?
>
> Well, you can't use the XNET protocol from a remote desktop
connection, even if the RDT client is running on the same physical
machine as the server. Your RDT client will need to use TCP/IP
protocol (servername:alias or servername:disk:\path\to\database_file)
>
> ./heLen
>