Subject Re: [firebird-support] Should autosweep be happening on that database?
Author Ivan Arabadzhiev
Well, first of all - thank you for your time.
Trace log still showed nothing but I finally noticed a pattern and talked to one of the devs (who didn`t know but has access to the source) - apparently I`m going to have to strangle someone who has never seen the size of a production database :)

2014-10-29 22:02 GMT+02:00 Thomas Steinmaurer ts@... [firebird-support] <firebird-support@yahoogroups.com>:
 

> Thanks. I`ve fixed the config and started a new trace session. Now we
> wait again :)

To shorten the time to wait to see if that works in general, you could
simply play with a test database using a similar alias thus database filter.

--
With regards,
Thomas Steinmaurer
http://www.upscene.com/

Professional Tools and Services for Firebird
FB TraceManager, IB LogManager, Database Health Check, Tuning etc.

> 2014-10-29 21:31 GMT+02:00 Thomas Steinmaurer ts@...
> <mailto:ts@...> [firebird-support]
> <firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>:
>
> __
>

> > Trace config :
> >
> > <database orbis-aton>
> > enabled true
> > print_perf true
> > log_sweep true
> > exclude_filter %RDB$%
> > time_threshold 0
> > </database>
> > <services>
> > enabled true
> > log_services true
> > include_filter "Repair Database"
> > </services>
>
> The database filter expects a regular expression and orbis-aton
> can't be
> parsed as a regular expression upon trace session start.
>
> Either escape the minus symbol resulting in:
>
> orbis\-aton
>
> or use % as a wildcard, e.g.:
>
> %orbis%
>
> This should now give you appropriate sweep trace events for the
> database
> trace configured in the <database> section.
>
> > Cron calls a simple bash script with the following command for each of
> > the databases :
> >
> > /opt/firebird/bin/gfix -user sysdba -password masterkey -sweep orbis-aton
>
> I see now that services tracing via the <services> section won't result
> in any trace data when executing the sweep through gfix, because this
> doesn't go through the services manager. And the new command-line tool
> fbsvcmgr doesn't seem to support the switch for sweeping.
>
> --
> With regards,
> Thomas Steinmaurer
> http://www.upscene.com/
>
> Professional Tools and Services for Firebird
> FB TraceManager, IB LogManager, Database Health Check, Tuning etc.
>
> >
> >
> > 2014-10-29 20:39 GMT+02:00 Thomas Steinmaurerts@... <mailto:ts@...>

> > <mailto:ts@... <mailto:ts@...>> [firebird-support]
> > <firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>>:
> >
> > __
> >
>
> > > Well, I`ve done two things since the last post
> > >
> > > 1 I set the sweep interval to 100000 (I assume I`m far from
> reaching
> > > that on a normal day)
> > > No change - Yesterday, another unexpected sweep at 5:08:03 pm
> > >
> > > 2 I started a trace session with the suggested config (changing
> > only DB
> > > alias, as instructed - I don`t really have much experience with the
> > > trace API)
> > > Today a sweep happened at 5:08:56 (about 28000 transactions
> after the
> > > morning manual sweep). It`s suspiciously close as a time frame, but
> > > looking back at the last month or so - sweeps are distributed
> in the
> > > 5-10pm range, so I guess it could be a coincidence.
> > >
> > > Trace log ti this moment is as follows :
> > > Trace session ID 2 started
> > > 2014-10-28T18:06:59.8670 (9756:0x7f5e704849e0) TRACE_INIT
> > > SESSION_2
> > >
> > >
> > > 2014-10-28T18:07:57.6530 (9768:0x7f7d2e4399e0) TRACE_INIT
> > > SESSION_2
> > >
> > >
> > > 2014-10-28T18:07:57.6530 (9768:0x7f7d2e4399e0) ATTACH_SERVICE
> > > service_mgr, (Service 0x7f7d2e4578d0, firebird, internal)
> > >
> > > 2014-10-28T18:07:57.6530 (9768:0x7f7d2e4399e0) DETACH_SERVICE
> > > service_mgr, (Service 0x7f7d2e4578d0, firebird, internal)
> > >
> > > 2014-10-28T18:07:57.6530 (9768:0x7f7d2e4399e0) TRACE_FINI
> > > SESSION_2
> > >
> > >
> > > 2014-10-28T18:08:16.7590 (9779:0x7fdb860fd9e0) TRACE_INIT
> > > SESSION_2
> > >
> > >
> > > 2014-10-28T18:08:16.7590 (9779:0x7fdb860fd9e0) ATTACH_SERVICE
> > > service_mgr, (Service 0x7fdb8611b8d0, firebird, internal)
> > >
> > > 2014-10-28T18:08:16.7590 (9779:0x7fdb860fd9e0) DETACH_SERVICE
> > > service_mgr, (Service 0x7fdb8611b8d0, firebird, internal)
> > >
> > > 2014-10-28T18:08:16.7590 (9779:0x7fdb860fd9e0) TRACE_FINI
> > > SESSION_2
> > > I assume those entries are from when I was poking around in the
> > > beginning. The trace session is still active but log says nothing
> > about
> > > sweeping. Wasn`t it supposed to show at least the sweep by gfix
> > (the one
> > > running from cron)? I`ll try fine tune it a bit on the weekend
> > when most
> > > users are gone and won`t notice. If anyone has any suggestions I
> > could
> > > test to help in the mean time - they`d be appreciated.
> >
> > Can you show us the changed trace config and the exact gfix call in
> > your
> > cron job?
> >
> > --
> > With regards,
> > Thomas Steinmaurer
> > http://www.upscene.com/
> >
> > Professional Tools and Services for Firebird
> > FB TraceManager, IB LogManager, Database Health Check, Tuning etc.
> >
> > >
> > > 2014-10-27 22:41 GMT+02:00 Thomas
> Steinmaurerts@...
> <mailto:Steinmaurerts@...> <mailto:ts@...
> <mailto:ts@...>>
>
> > > <mailto:ts@... <mailto:ts@...>
> <mailto:ts@... <mailto:ts@...>>>
> > [firebird-support]
> > > <firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>
> > > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>>>:
> > >
> > > __
> > >
> > > > I`m not sure if running a 24/7 trace is a good idea (database
> > is under
> > > > relatively heavy load). Anything else I could do to help?
> > >
> > > It might be ok with a proper include filter at service level from a
> > > Trace API POV.
> > >
> > > The following trace configuration should be fully dedicated to
> > sweeping
> > > only:
> > >
> > > <database yourdatabasealias>
> > > enabled true
> > > print_perf true
> > > log_sweep true
> > > exclude_filter %RDB$%
> > > time_threshold 0
> > > </database>
> > > <services>
> > > enabled true
> > > log_services true
> > > include_filter "Repair Database"
> > > </services>
> > >
> > > --
> > > With regards,
> > > Thomas Steinmaurer
> > > http://www.upscene.com/
> > >
> > > Professional Tools and Services for Firebird
> > > FB TraceManager, IB LogManager, Database Health Check, Tuning etc.
> > >
> > > > btw I am noticing that the unexpected sweep is about 27000-30000
> > > > transactions after the scheduled one. I could send a log if
> > that helps
> > > > in any way. I set the interval to 100000 and in a day or two
> > I`ll know
> > > > if it still runs by its own.
> > > >
> > > > 2014-10-27 21:46 GMT+02:00 Thomas
> > Steinmaurerts@...
> <mailto:Steinmaurerts@...>
> > <mailto:Steinmaurerts@...

> <mailto:Steinmaurerts@...>> <mailto:ts@...
> <mailto:ts@...>
> > <mailto:ts@... <mailto:ts@...>>>
> > > > <mailto:ts@... <mailto:ts@...>
> <mailto:ts@... <mailto:ts@...>>
>
> > <mailto:ts@... <mailto:ts@...>
> <mailto:ts@... <mailto:ts@...>>>>
> > > [firebird-support]
> > > > <firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>
> > > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>>
> > > > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>
> >
> > > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>
> > <mailto:firebird-support@yahoogroups.com
> <mailto:firebird-support@yahoogroups.com>>>>>:
> > > >
> > > > __
> > >
> > >
> > > >
> > > > > I did some reconfiguration on a database yesterday, so I`m
> > > monitoring
> > > > > the logs for any issues and I noticed something strange :
> > > > >
> > > > > Mon Oct 27 16:54:25 2014
> > > > > Sweep is started by SYSDBA
> > > > > Database "orbis-aton"
> > > > > OIT 21080, OAT 21081, OST 20456, Next 44177
> > > > >
> > > > > (yes, I know it`s a bad idea but all database connections
> are by
> > > > SYSDBA)
> > > > >
> > > > > I don`t remember running a sweep (logs say I wasn`t even logged
> > > in at
> > > > > the time) and nobody else has access (or knowledge, for that
> > > > matter) to
> > > > > run a sweep on that machine so I got curious.
> > > > > Logs say an unexpected sweep has been running basically
> every day
> > > > > (sometime between 5 and 10 pm, which is apparently when it
> > hits the
> > > > > number of transactions). I have a manual sweep at 5:40am in
> cron
> > > > and I
> > > > > checked - configuration is correct and runs as expected. I also
> > > > > double-checked gstat :
> > > > >
> > > > >
> > > > > Database header page information:
> > > > > Flags 0
> > > > > Checksum 12345
> > > > > Generation 51454
> > > > > Page size 16384
> > > > > ODS version 11.2
> > > > > Oldest transaction 21357
> > > > > Oldest active 50561
> > > > > Oldest snapshot 49520
> > > > > Next transaction 51318
> > > > > Bumped transaction 1
> > > > > Sequence number 0
> > > > > Next attachment ID 130
> > > > > Implementation ID 24
> > > > > Shadow count 0
> > > > > Page buffers 0
> > > > > Next header page 0
> > > > > Database dialect 3
> > > > > Creation date Oct 26, 2014 12:28:26
> > > > > Attributes force write
> > > > >
> > > > > Variable header data:
> > > > > Sweep interval: 0
> > > > > *END*
> > > > >
> > > > >
> > > > > Shouldn`t automatic sweep be disabled or have I misread the
> > > > > documentation? FB version is 2.5.3.
> > > >
> > > > Yes, by setting the sweep interval to 0, automatic sweeping
> > should be
> > > > disabled. There is also:
> > > > http://tracker.firebirdsql.org/browse/CORE-4100
> > > > (marked as fixed in 2.5.3) but you are mentioning that you are
> > using
> > > > 2.5.3, although I'm not entirely sure if CORE-4100 applies to
> sweep
> > > > interval = 0 anyway.
> > > >
> > > > You could use the Trace API to trace services API requests to
> get a
> > > > clearer picture on what client application is causing the sweep
> > > in case
> > > > it has been triggered manually.
> > > >
> > > > --
> > > > With regards,
> > > > Thomas Steinmaurer
> > > > http://www.upscene.com/
> > > >
> > > > Professional Tools and Services for Firebird
> > > > FB TraceManager, IB LogManager, Database Health Check, Tuning
> etc.
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
>
>
>
>
>