Hi Geoff,

Since D5 doesn't seem to have a definition for soEnd,
I replaced
tmpRead.Seek(0, soEnd);
with
tmpRead.Seek(0, soFromBeginning);

Hoping this is the same, now IBOjects using TDataPump
is quite a bit faster.

Best Regards
Hans

----- Original Message -----
From: "Geoff Worboys" <geoff@...>
To: <IBObjects@yahoogroups.com>
Sent: Tuesday, March 23, 2010 1:23 AM
Subject: Re: [IBO] development


> Hi Jason and Hans,
>
>
> Jason, can you please check that this code looks acceptable
> (the lines marked //??TC are the ones added/modified by me):
>
>
> procedure TIB_ColumnBlob.LoadFromBlob( const ABlob: TIB_ColumnBlob );
> var
> tmpStream: TStream;
> tmpRead: TIB_BlobStream; //??TC
> bn: PIB_BlobNode;
> begin
> if ( not Assigned( ABlob )) or ( ABlob.IsNull ) then
> Clear
> else
> begin
> tmpStream := Statement.CreateBlobStream( Self, bsmWrite );
> tmpRead := ABlob.Statement.CreateBlobStream(ABlob, bsmRead); //??TC
> try
> if not BlankIsNull then
> SysSetIsNull( false );
> //??TC Old Code: ABlob.SaveToStream( tmpStream );
> //??TC Optimised blob copy - read all and then copy buffer entire
> //??TC (rather than constant resizing that goes on with stream copy
> tmpRead.Seek(0, soEnd); //??TC
> bn := tmpRead.BlobNode; //??TC
> if Assigned(bn) and Assigned(bn.BlobBuffer) and (bn.BlobSize > 0)
> then //??TC
> tmpStream.WriteBuffer(bn.BlobBuffer^, bn.BlobSize); //??TC
> finally
> tmpRead.Free; //??TC
> tmpStream.Free;
> end;
> end;
> end;
>
>
> Note that SaveToBlob will need the equivalent optimisation.
>
>
> The problem with the original SaveToStream call is that it
> calls the default TStream.CopyFrom:
>
> function TStream.CopyFrom(Source: TStream; Count: Int64): Int64;
> const
> MaxBufSize = $F000;
>
> For blobs larger than $F000 we end up calling IB_ReallocMem
> for as many times as it takes to grow to the required size in
> $F000 increments (very many reallocations and memory copies).
>
> The new code takes advantage of the fact that CreateBlobStream
> actually ends up loading the entire blob into a memory buffer
> and so we can easily write a copy of that buffer directly with
> no need for realloc etc. (It would be tempting to try and
> implement a shared buffer here so that no copy was required at
> all - until/unless one of the owners attempted to write.)
>
> The current code is still not ideal for high performance
> transfer situations (in DBak I wrote specialised blob-copy that
> went directly from field to field, never loading the entire
> blob) but the change is certainly an improvement and incurs no
> down-side (since the TStream.CopyFrom results in exactly the
> same blob load, but with the above code we can take advantage
> of the more direct access).
>
>
> The result after this change was:
>
> = = = xfer_ibo = = =
> App Init Time: 0ms
> Object Prepare Time: 109ms
> Records Transfered: 106
> Records Transfer Time: 10484ms
> Transaction Commit Time: 1188ms
> Connection Close Time: 0ms
> Application Run Time: 11781ms
>
> (Over several iterations both IBO and FIBPlus come out around
> the same sort of average for this table now... would probably
> take a longer test to offer a more accurate comparison.)
>
>
> Jason, additional (less substantial) gains are possible via
> more analysis of IB_ReallocMem - as this can be an expensive
> call in large buffer situations. For example in
> function TIB_SessionBase.SysGetBlobData(
> about 13 lines up from the bottom is the line:
> if BlobNode.BlobBufLen > TotalSize then
> Due to the previous logic this test is almost always true and
> so the subsequent IB_ReallocMem is almost always called - and
> yet it is rarely necessary (the intentional excess allocated
> previously is for an unusual situation rather than usual).
> It seems that there should be a better way to deal with this.
> (Commenting out this bloc gave me, approximately, an extra
> second on the test... not that big but not nothing :-)
>
> --
> Geoff Worboys
> Telesis Computing
>
>
>
> ------------------------------------
>
> ___________________________________________________________________________
> IB Objects - direct, complete, custom connectivity to Firebird or
> InterBase
> without the need for BDE, ODBC or any other layer.
> ___________________________________________________________________________
> http://www.ibobjects.com - your IBO community resource for Tech Info
> papers,
> keyword-searchable FAQ, community code contributions and more !
> Yahoo! Groups Links
>
>
>