Subject | Updating BLOBS and speed degradation |
---|---|
Author | nnrpropsoft |
Post date | 2003-11-09T11:03:44Z |
I use the TIB components , TIB_Connection , TIB_Transaction and
TIB_Cursor in my Delphi 6 app.
Using Firebird 1.0
(machine = 256mb RAM 1.8Ghz AcerLaptop) as opposed to the LAN which
is where the app is supposed to run (typically 100mbps).
i update a BLOB field with files often exceeding 10 or 20 megs
(potentially up to 20 gig), but these hang my machine, utilising all
cpu resources
I use the ".LOADFROMFile" Method
I require some other faster means of populating BLOBs
When updating a BLOB field with smaller objects (zipped files in this
case), 2 megs
the .LoadFromFile method executes in some 5 seconds and
the POST method takes +- 3 seconds
but
when the size of the file being updated to the blob field gets to 20
times that ie 40 megs
the .LoadFromFile method executes in some 420 seconds
(ie 84 times longer than a file 20 times smaller)
the POST method takes +- 70 seconds (slightly more than 20 times
as long as 2megs)
I accept that physically writing to the dbase would take some time
(HDD speeds etc) but is there
another method I could use, which would not utilise all my machines
memory , and take substantially quicker to execute ?
I have include the code I use below.
Thanks, much appreciated.
Neal Rogers
function TDoBackForm.BackupFiles(BackId: Integer; var BackupSize:
integer; x: integer; WFileName, DirectoryOnly: string):TActionResult;
var
WCursor: TIB_Cursor;
FileSize: Integer;
WTimeStamp: TDateTime;
begin
Result.ErrorCode := 0;
Result.ResultString := '';
WCursor := TIB_Cursor.Create(Self);
WTimeStamp:= now;
try
try
WCursor.SQL.Text := 'SELECT * FROM BACKFILE FOR UPDATE';
WCursor.Append;
WCursor.FieldByName('ID').AsInteger := WCursor.GeneratorValue
('BACKFILE_ID', 1);
WCursor.FieldByName('BACKUP').AsInteger := BackId;
WCursor.FieldByName('FILETIME').AsDateTime := WTimeStamp;
WCursor.FieldByName('FILENAME').AsString := WFileName;
WCursor.FieldByName('DIRECTORYONLY').AsString := DirectoryOnly;
TIB_ColumnBlob(WCursor.FieldByName('BACKUPDATA')).LoadFromFile
(ExtractFilePath(Application.ExeName)+'tmp'+IntToStr(x)+'.enc');
WCursor.Post;
except
Result.ResultString := 'Error: BackupFiles to DataBase';
Result.ErrorCode := -2;
end;
finally
WCursor.Free;
end;
end;
TIB_Cursor in my Delphi 6 app.
Using Firebird 1.0
(machine = 256mb RAM 1.8Ghz AcerLaptop) as opposed to the LAN which
is where the app is supposed to run (typically 100mbps).
i update a BLOB field with files often exceeding 10 or 20 megs
(potentially up to 20 gig), but these hang my machine, utilising all
cpu resources
I use the ".LOADFROMFile" Method
I require some other faster means of populating BLOBs
When updating a BLOB field with smaller objects (zipped files in this
case), 2 megs
the .LoadFromFile method executes in some 5 seconds and
the POST method takes +- 3 seconds
but
when the size of the file being updated to the blob field gets to 20
times that ie 40 megs
the .LoadFromFile method executes in some 420 seconds
(ie 84 times longer than a file 20 times smaller)
the POST method takes +- 70 seconds (slightly more than 20 times
as long as 2megs)
I accept that physically writing to the dbase would take some time
(HDD speeds etc) but is there
another method I could use, which would not utilise all my machines
memory , and take substantially quicker to execute ?
I have include the code I use below.
Thanks, much appreciated.
Neal Rogers
function TDoBackForm.BackupFiles(BackId: Integer; var BackupSize:
integer; x: integer; WFileName, DirectoryOnly: string):TActionResult;
var
WCursor: TIB_Cursor;
FileSize: Integer;
WTimeStamp: TDateTime;
begin
Result.ErrorCode := 0;
Result.ResultString := '';
WCursor := TIB_Cursor.Create(Self);
WTimeStamp:= now;
try
try
WCursor.SQL.Text := 'SELECT * FROM BACKFILE FOR UPDATE';
WCursor.Append;
WCursor.FieldByName('ID').AsInteger := WCursor.GeneratorValue
('BACKFILE_ID', 1);
WCursor.FieldByName('BACKUP').AsInteger := BackId;
WCursor.FieldByName('FILETIME').AsDateTime := WTimeStamp;
WCursor.FieldByName('FILENAME').AsString := WFileName;
WCursor.FieldByName('DIRECTORYONLY').AsString := DirectoryOnly;
TIB_ColumnBlob(WCursor.FieldByName('BACKUPDATA')).LoadFromFile
(ExtractFilePath(Application.ExeName)+'tmp'+IntToStr(x)+'.enc');
WCursor.Post;
except
Result.ResultString := 'Error: BackupFiles to DataBase';
Result.ErrorCode := -2;
end;
finally
WCursor.Free;
end;
end;