Subject | Re: [firebird-support] Re: Strange behavior on very large table |
---|---|
Author | Lester Caine |
Post date | 2019-02-22T09:38:04Z |
On 22/02/2019 04:33, myles@... [firebird-support] wrote:
be in the one table? Can you re-configure so that this action becomes
'add new table' ... 'drop old table' ... which will be a lot tidier anyway.
I had a system loading a previous days actions which initially just kept
adding and keeping history, and with a more modern processor and SSD
disk we would probably get away with it today ;) But back then it got to
a point where the table was too big, and we dropped to just maintaining
a months worth live with every day in it's own table and only the
summary tables maintaining the historic counts. All the historic data is
still available and if there was a need it might be interesting to put
it all back together into one table ... if I could get past the 'data
protection' restrictions :( I am actually expecting a request to make
sure it is all destroyed but it's not directly identifying people from
20 years ago.
--
Lester Caine - G8HFL
-----------------------------
Contact - https://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - https://lsces.co.uk
EnquirySolve - https://enquirysolve.com/
Model Engineers Digital Workshop - https://medw.co.uk
Rainbow Digital Media - https://rainbowdigitalmedia.co.uk
> >Given that this table is "temporary" storage, one supposes that youHow much information is static between cycles? Does all the data NEED to
> are deleting rows from it regularly. Do you happen to be deleting
> 900,000 rows each day before you load up the latest batch of 900,000?
be in the one table? Can you re-configure so that this action becomes
'add new table' ... 'drop old table' ... which will be a lot tidier anyway.
I had a system loading a previous days actions which initially just kept
adding and keeping history, and with a more modern processor and SSD
disk we would probably get away with it today ;) But back then it got to
a point where the table was too big, and we dropped to just maintaining
a months worth live with every day in it's own table and only the
summary tables maintaining the historic counts. All the historic data is
still available and if there was a need it might be interesting to put
it all back together into one table ... if I could get past the 'data
protection' restrictions :( I am actually expecting a request to make
sure it is all destroyed but it's not directly identifying people from
20 years ago.
--
Lester Caine - G8HFL
-----------------------------
Contact - https://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - https://lsces.co.uk
EnquirySolve - https://enquirysolve.com/
Model Engineers Digital Workshop - https://medw.co.uk
Rainbow Digital Media - https://rainbowdigitalmedia.co.uk