Subject | RE: [firebird-support] Re: Fixing Database |
---|---|
Author | Alan McDonald |
Post date | 2005-03-14T09:45:42Z |
> Thanks all,read up on the FIRST SKIP syntax - it's the same result as limit in MySQL.
>
> I will start with changing the protocol to TCP.
> The path to the DB is surely correct and no more than one connection
> uses it. I see this by dumping the ADO connection string to the log,
> from where I took the example.
> Selects are not so huge, as records are processed by first letter, so
> thay can be as large as 200K rows. Then the app identifies
> problematic records (wrong formatted or duplicated data, for
> instance) and presents these records for editing by human. Yes, they
> sometimes need to manually edit hundreds of records, but this is the
> legacy from their old application.
> Once the "letter" processing is over, the application re-selects the
> records from the temp data (usually less than the original number),
> generates in memory, for each record some additional data and inserts
> into the 4 tables I mentioned. There is one big SELECT, then
> itteration thru the records and inserts into the new tables.
>
> I've been running tests several times on a new, test DB since the
> crash occurs and cannot reproduce the situation. I also see that the
> temp files are being deleted properly.
>
> Now, to a possible reason for the crash...
> We've found today that near the time of the crash, the anti-virus
> program, AVG, performed automatic update, downloaded files to the
> same Windows TEMP directory, installed them and restarted itself.
> I've already contacted them for details about this operation. We will
> also try to simulate it, to see if this can be the source of the
> problem.
>
> Now, another question, related to the big selects, discussed here.
> Supposed I know that the select may return high number of records
> (thousands and more). How can I limit the number of records returned?
> Get them by predfined blocks? Is there something similar to the LIMIT
> clause (as in MySQL?). I've been looking in the Interbase
> documentation, and seems the only answer is to use CURSOR. Or am I
> wrong? Sorry, but as I said, I'm newbie to FB/IB.
>
But the better method by far is based on the normalised data hierarchy if
you have one. Parameterised queries in a client side setup, or FOR SELECT
INTO loops on the server. But the data must be hierarchical to achieve this.
Alan
PS - I doubt very much the autoupdate of NAV will have anything to do with
your issue.
as for ways to improve your overall performance? I would drop the ADO/ODBC
connection stuff as soon as you can, there are several far beter ways to
interact with your data than that. You've never said what language you are
using.
Alan