| Subject | Re: [ib-support] large deletions |
|---|---|
| Author | Jason Chapman (JAC2) |
| Post date | 2002-07-02T17:48:26Z |
Thoughts on high availability ticker DB's (that's what I call them):
Firstly for a 24 x 7 x 30.8 I would normally have 2 db's running in
parallel. That way you have additional fault tolerance and can run a
bespoke replication to allow inserts into either DB to propagate to the
other (this is so application specific, I've done it for one mfr co and
advised at least one other).
This way if you need to you can take a DB down you can ensure it is
non-live, then let the replication deplete, then it is yours to do whatever
with.
On the deletion I would:
1) Delete 1000 rows at a time then do a commit. You need to do a select
that would attempt to touch these records, otherwise you will still be
saving up the garbage collection. This generates minimum CPU stress as a
1000 records to garbage collect is nothing.
2) You have no problems with doing this in a separate thread of an
application (as you asked), as long as this thread is completely isolated
(makes its own connection to the DB etc).
3) Ensure that any app disconnects at least once every 24 hours, or ensure
you have bullet proof code that doesn't leave transactions open for ages.
This really kills the DB performance if the DB can't successfully garbage
collect 200,000 x 30 odd days of deletes.
4) Try and avoid the lazy coding of doing data changes and rollback,
rollbacks with data changes can cause problems for IB / FB if you push it to
its limits in terms of transaction insertions , updates and deletions
without sweeps.
My last place had a DB that took a couple of hundred users x 000,000 inserts
x 30days and starts to slow down and increase CPU usage.
Anyway, I would probably look at the requirements and decide on whether
making a 24 x 7 x 365 x years to your retirement architecture with multiple
DB's makes sense. It's not that expensive if you have a couple of machines
etc.
Cheers,
Jason Chapman
JAC2 Consultancy
Shameless plug ....
Training - Development - Consultancy
Delphi, InterBase, Firebird, OOAD, Development lifecycle assistance,
Troubleshooting projects, QA.....
www: When I get round to it....
Mob: (+44) 07966 211 959 (preferred)
Tel: (+44) 01928 751088
""Bob Lazarchik"" <bob5407@...> wrote in message
news:004b01c221d7$13526f50$5900000a@bob1...
Firstly for a 24 x 7 x 30.8 I would normally have 2 db's running in
parallel. That way you have additional fault tolerance and can run a
bespoke replication to allow inserts into either DB to propagate to the
other (this is so application specific, I've done it for one mfr co and
advised at least one other).
This way if you need to you can take a DB down you can ensure it is
non-live, then let the replication deplete, then it is yours to do whatever
with.
On the deletion I would:
1) Delete 1000 rows at a time then do a commit. You need to do a select
that would attempt to touch these records, otherwise you will still be
saving up the garbage collection. This generates minimum CPU stress as a
1000 records to garbage collect is nothing.
2) You have no problems with doing this in a separate thread of an
application (as you asked), as long as this thread is completely isolated
(makes its own connection to the DB etc).
3) Ensure that any app disconnects at least once every 24 hours, or ensure
you have bullet proof code that doesn't leave transactions open for ages.
This really kills the DB performance if the DB can't successfully garbage
collect 200,000 x 30 odd days of deletes.
4) Try and avoid the lazy coding of doing data changes and rollback,
rollbacks with data changes can cause problems for IB / FB if you push it to
its limits in terms of transaction insertions , updates and deletions
without sweeps.
My last place had a DB that took a couple of hundred users x 000,000 inserts
x 30days and starts to slow down and increase CPU usage.
Anyway, I would probably look at the requirements and decide on whether
making a 24 x 7 x 365 x years to your retirement architecture with multiple
DB's makes sense. It's not that expensive if you have a couple of machines
etc.
Cheers,
Jason Chapman
JAC2 Consultancy
Shameless plug ....
Training - Development - Consultancy
Delphi, InterBase, Firebird, OOAD, Development lifecycle assistance,
Troubleshooting projects, QA.....
www: When I get round to it....
Mob: (+44) 07966 211 959 (preferred)
Tel: (+44) 01928 751088
""Bob Lazarchik"" <bob5407@...> wrote in message
news:004b01c221d7$13526f50$5900000a@bob1...
>are
>
> > Hi Bob,
> >
> > BL> The constraints are that this tool must continue to run during the
> > BL> clean up process.
> >
> > Does that tool run 24/24 7/7?
>
> Yes. This tool does metrology on Silicon wafers used in making IC's and
> runs 24/24 7/7. There are allowed Preventive Maintenance times but they
> only available monthly and less than 4 hours for all maintenance tasks.
>
> Bob
>
>
>
> To unsubscribe from this group, send an email to:
> ib-support-unsubscribe@egroups.com
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
>