i actually tried to load the data into a memory table, but it was over 2 gigs..
Maybe a simple array would be better..

----- Original Message -----
From: Michael Horne
To: IBObjects@yahoogroups.com
Sent: Tuesday, August 26, 2008 9:32 PM
Subject: RE: [IBO] Performance..


Hello,

If you are using Delphi, here are some things you can try

Connection.BeginBusy(True);
query.DisableControls;
try
---- process code here
finally
Connection.EndBusy;
query.EnableControls;
end;

Also, I may have my numbers off but does the data take up
about 250MB, if so then why not create a large set of arrays
and just read the db once to load the arrays and then process
it out of memory.

Just some thoughts.

Good Luck
Michael Horne


_____

From: IBObjects@yahoogroups.com [mailto:IBObjects@yahoogroups.com] On Behalf
Of Gordon Hamm
Sent: Tuesday, August 26, 2008 6:29 PM
To: IBObjects@yahoogroups.com
Subject: Re: [IBO] Performance..

Thanks all for the help..

Im not sure what Affinity is..How can I set that?

Im doing a select everytime, because I have the query "Unidirectional" set
to True.. When I had it set to false, I was using lots of memory.. I assumed
that since it was sent to unidirectional, I had to requery everytime..

Is this right?

----- Original Message -----
From: Robert martin
To: IBObjects@yahoogrou <mailto:IBObjects%40yahoogroups.com> ps.com
Sent: Tuesday, August 26, 2008 2:04 PM
Subject: Re: [IBO] Performance..

Hi

If you are using Superserver have you set the CPU Affinity (otherwise
multicores can sometimes degrade performance). I assume you are only
running the query once and then just looking through the results
multiple times? If so is it that the performance issue is with the
processing code not FB? You could test this by just commenting out your
processing code and just leaving in the looping. If the speed
performance is not dramatically better I would guess the problem is with FB.

Rob

Gordon Hamm wrote:
> The funny thing about all this, is that the hard drive is hardly working..
and the cpu is only at about 20%.
>
> Its like Firebird doesnt know how to utilize the machine..
>
> ----- Original Message -----
> From: Aage Johansen
> To: ibobjects@yahoogrou <mailto:ibobjects%40yahoogroups.com> ps.com
> Sent: Tuesday, August 26, 2008 12:24 PM
> Subject: Re: [IBO] Performance..
>
>
> Gordon Hamm wrote:
> > Hi,
> > Im using Firebird 2.1.
> > I have a table that has about 8 fields of prices and a date/timestamp
field.
> > There are about 7 million records.
> > There is only one index, and thats on the time stamp field.
> > I have a routine that does a simple sql select and orders on the
date/time
> > field (That has an index)
> >
> > I then loop through the entire table, and do some complex math etc on
the
> > data.. top to bottom..
> > Then, I do it again and again, testing some modeling parameters or
> "What if"
> > stuff. I will litterally loop through it 500 times..
> >
> > No, there is no way to do this in a stored proc..
> >
> > Anyway, It works fine, but I was doing it on my pentium 4, took about 5
> > minutes for each iteration.
> > memory usage was small, and CPU was only about 30 %.
> >
> > I went out and bought a quad core , with more memory, much faster cpu
and
> > bus, faster drives etc..
> >
> > Its only a tiny bit faster now.. Im so discouraged with it.. I can
imagine
> > why its not many times faster..
> >
> > Any idea what the bottle neck might be? I am doing a unidirection query,
to
> > save memory, that helps, Im not using "Fieldbyname", getting the field
> > directly by TempQuery.Fields.Fields[12].AsFloat.
> >
> > I dont know what else to do..
> >
> > Gordon
> >
>
> If you do
> Select * from HedgeData Order By TradeDateTime
> on 7 million rows, you should note that ordering on an indexed field
> isn't necessarily a good thing. Firebird will be chasing records in
> index order which may mean a lot of disk arm movement. Try to order
> without using the index. If you don't want to drop the index you
> could try this version of the select:
> Select * from HedgeData Order By TradeDateTime+0
> Firebird wil (probably) fetch all rows in storage order, and then
> apply a sort (which usually is quite fast).
>
> --
> Aage J.
>
>
>
>
>
> [Non-text portions of this message have been removed]
>
>
> ------------------------------------
>
> __________________________________________________________
> IB Objects - direct, complete, custom connectivity to Firebird or
InterBase
> without the need for BDE, ODBC or any other layer.
> __________________________________________________________
> http://www.ibobject <http://www.ibobjects.com> s.com - your IBO community
resource for Tech Info papers,
> keyword-searchable FAQ, community code contributions and more ! Yahoo!
Groups Links
>
>
>
>
>

[Non-text portions of this message have been removed]

[Non-text portions of this message have been removed]





[Non-text portions of this message have been removed]