Subject thoughts on duplicate checking, input needed
Author Thomas Besand
Hi NG,

first of all: thanks for the fantastic support you provide; it's always
a pleasure to follow the discussions and threads in this forum.
Appreciate it a lot!

This said, I want to come up with a question regarding my current project.
I'm implementing a database of customer adresses. Tools applied: FB
1.5.2, Delphi 5 with Zeos and IBX components.
The product is nearing its final development stage; all I want to put in
now is functionality to merge tables that are already in the db and
carry out some sort of duplicate checking in the merge process.
More details: I have several hundred tables with adresses (tables do not
have identical structures) and want to merge selected tables with
others. I give the user a form where he/she can adjust column
correlations, i.e. select which column from the source table will go
into which column into the destination table.
After finishing this, the merge process is started and is supposed to
check whether the values in the source columns are identical with the
corresponding columns in the destination table. If identical values are
detected, the program shall update the existing record in the
destination table with the values from the source table; if not append
the record to the destination table.
After thinking about this process for some days I fear now that this
functionality will grab quite some resources and take a long time to
run, as I think the process should follow this sequence:
1. grab the values in question from the source table
2. try to find identical values in the destination table
3. if found, update destination table
4. if not found, append to destination table
5. grab next record from source table

Maybe anyone can provide some thoughts about this and come up with an
idea how to elegantly implement this function.

Thanks beforehand
Greets
Thomas Besand, Berlin