Subject | Re: sweep rule of thumb timings |
---|---|
Author | csswa |
Post date | 2002-04-01T16:12:15Z |
--- In ib-support@y..., "Jason Chapman (JAC2)" <jason@j...> wrote:
across to the old system (bedroom to loungeroom <grin>) in a minute
or two. And I can gbak the whole 100+meg db in about 2 min so it's
unlikely to be an I/O bottleneck.
I don't have any large db files to crunch for comparisons -- to see
if it (main NT system) sweeps them all at 100k/sec. no, wait I do...
yep, just swept a 50,000-record 25-meg file and it swept at about
600k/sec. This particular file has an old production table as the
bulk of it; it wasn't autogenerated by a data-filler like my 100meg
test file. It has six individual-field indexes and no primary key.
Hmm, wonder what happens if I add a primary key...
Just tried that. Added primary key and no sweep speed change from
above. Deleted other indexes and ditto.
Also FWIW, regarding threads, I noticed that my normal ibserver
process under NT has 4 unnamed threads and one 'ib server' thread.
Under Win95 there is just one unnamed thread and the 'ib server'
thread. When gfix-sweeping under NT, this jumps to 8 threads in
total (2 for gfix and one extra for the sweep[?]).
So at this point I can only attribute the slow 100k/sec sweep speed
on my 100meg test db to the data contained in it. There has to be
something weird about having a big table (1 million records,
remember) with a series of low-select fields (range of about 100200
unique values for each field). This would make sense if there were
indexes other than the one integer primary key *and* the db actually
had garbage to collect...
Thanks, Jason.
Regards,
Andrew Ferguson
> What rates do you get for other non-IB processes.Well all other file I/O seems normal. The actual db file copied
> E.G. normal file to and from the same disk on each.
> Some kind of memory allocation de-allocation
> + a number crunching testg?
across to the old system (bedroom to loungeroom <grin>) in a minute
or two. And I can gbak the whole 100+meg db in about 2 min so it's
unlikely to be an I/O bottleneck.
I don't have any large db files to crunch for comparisons -- to see
if it (main NT system) sweeps them all at 100k/sec. no, wait I do...
yep, just swept a 50,000-record 25-meg file and it swept at about
600k/sec. This particular file has an old production table as the
bulk of it; it wasn't autogenerated by a data-filler like my 100meg
test file. It has six individual-field indexes and no primary key.
Hmm, wonder what happens if I add a primary key...
Just tried that. Added primary key and no sweep speed change from
above. Deleted other indexes and ditto.
Also FWIW, regarding threads, I noticed that my normal ibserver
process under NT has 4 unnamed threads and one 'ib server' thread.
Under Win95 there is just one unnamed thread and the 'ib server'
thread. When gfix-sweeping under NT, this jumps to 8 threads in
total (2 for gfix and one extra for the sweep[?]).
So at this point I can only attribute the slow 100k/sec sweep speed
on my 100meg test db to the data contained in it. There has to be
something weird about having a big table (1 million records,
remember) with a series of low-select fields (range of about 100200
unique values for each field). This would make sense if there were
indexes other than the one integer primary key *and* the db actually
had garbage to collect...
> Finally, and don't shout, the NTFS partition isn't compressed is it?No compression. :-)
Thanks, Jason.
Regards,
Andrew Ferguson
>second
> Cheers,
>
> JAC.
> ""csswa"" <csswa@y...> wrote in message
> news:a89qhs+r09v@e...
> > --- In ib-support@y..., "csswa" <csswa@y...> wrote:
> > > sweeps of the db via gfix (no other connections) take the same
> > time:
> > > 16 min. Watching the server process I can see the db file being
> > > walked sequencially from beginning to end at about 100k per
> > second.
> > > This contrasts with a backup which rips along at 1 meg per
> > > (gbak with default settings)!drive
> >
> > Hmm, the plot thickens. I copied the 100+meg db to my second
> > firebird 1.0 installation: a 333mhz win95 setup with old 1 gig
> > and 128mb ram... and the sweep ran at about 300k per second...this
> > compared to my main 850mhz 512meg 20gig hd NTsp6 setup of 100k permy
> > second!! On the NT system I have been careful to have minimal
> > processes running, no virus checker, etc. It's very strange that
> > old machine can run the sweep 3 times faster than my workhorsenone)
> > machine! The only difference I can see right now is the platform.
> > I'll keep whittling away to see if I can isolate the conditions.
> >
> > Regards,
> > Andrew Ferguson
> >
> >
> > >
> > > Gfix sweeping is a lot lighter on system resources (virtually
> > > than gbak, though.wrote:
> > >
> > > Header reveals:
> > >
> > > Page size 4096
> > > ODS version 10.0
> > > Oldest transaction 287
> > > Oldest active 288
> > > Oldest snapshot 288
> > > Next transaction 289
> > >
> > > So no weirdness there.
> > >
> > > Regards,
> > > Andrew Ferguson
> > >
> > >
> > > --- In ib-support@y..., "Ann W. Harrison" <aharrison@i...>
> > > > No, sweeping and garbage collect eliminate both the recordcheap,
> > > > and references to it in indexes. It finds the records by
> > > > walking through the tables sequentially (natural order).
> > > >
> > > > To remove entries from indexes, the code gets the value from
> > > > the record version it's removing, checks that the values isn't
> > > > used in a version of the record that's staying (reasonably
> > > > unless you do something like modifying the same record 500applies
> > > > while some transaction is open). If the value no longer
> > > > to the record, Firebird/InterBase goes to the index and looksfor
> > > > the value paired with the record id. Only values propagateup in
> > > > the index, so it must examine all entries in the index withthat
> > > > value until it finds the one with the matching record id.Here's
> > > > the nasty bit. Duplicates are stored at the front of thechain,
> > > > so the oldest records are last. Walking a table in naturalorder
> > > > removes the oldest records first.http://docs.yahoo.com/info/terms/
> > > >
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Ann
> > > > www.ibphoenix.com
> > > > We have answers.
> >
> >
> >
> > To unsubscribe from this group, send an email to:
> > ib-support-unsubscribe@e...
> >
> >
> >
> > Your use of Yahoo! Groups is subject to
> >
> >
> >