Subject | Re: INSERT Speed Optimization Ideas |
---|---|
Author | Adam |
Post date | 2005-06-05T21:08:04Z |
>Hi Robert,
> My question is (a) does this sound like a good solution for speeding
> up my inserts and (b) are there hidden gotchas in this solution that I
> am not considering?
>
> This table doesn't really have indices besides it's primary key so
> that approach really wont impact performance, and even if it did I
> don't think the insert performance change would be as drammatic as
> this solution.
>
I would add a flag to your table to specify which field the data is
stored in, rather than testing for NULL.
Also (based on your second posting), why have you declared your field
to be 2000 characters if you only ever store 1000 characters in it?
You will obviously need to do some tests on your data shape (eg. Max
field length required to date, average field length to date, and some
sort of graph of these). You can then do some experiments to see the
impact of different field sizes on the total database size and the
insert performance.
You may want to also consider changing the table structure a bit so
that perhaps you use a second table.
Table2
(
ID integer,
Sequence integer,
MyData varchar(whatever size is "optimum")
)
Then each record in your main table could store a table2ID, join to
table2 and run an order by on the sequence. The server could query all
records in Table2 and rebuild its data array. This could be done quite
efficiently, whether it will be faster or slower is not an academic
discussion, you will need to test it to see if it is worthwhile.
Secondly, consider on the fly compression if you haven't already. You
should be able to find some low resource requirement compressor that
achieves significant reduction in size.
Hope that is some ideas
Adam.