Subject RE: [firebird-support] Re: How to update table structure with SQL?
Author Jarrod Hollingworth
> The idea is run a script to update ANY database to the
> current version, no matter what the customer has.

You have many databases out there for different customers, and you don't
know what their database looks like, right? What you need is a consistent
approach to database versioning. Here's what to do:

a. Do a metadata and data export from your database (e.g. using SQL Manager
for Firebird).
b. Save this SQL script. It's the current version baseline that you will
compare with later.
c. Add a new value somewhere in the database that stores the current
database version number. I have a special system table for my app with
id/name/value columns and have a row with name='DBVER' but you could have a
simple one column, one row table to store it.
d. Set the database version to 0.

When you make changes to the database:
e. Do another metadata and data export.
f. Compare the new SQL script to the old one (use a text diff tool).
g. For each difference create one or more SQL statements to make the
appropriate change, such as adding columns, dropping tables, inserting or
updating rows, adding grants, updating triggers, and so on, adding them to
an ever growing "migration" SQL script.
h. Number each SQL statement. Add a /*VER=n*/ comment before each, starting
at n=1 and incrementing for each statement.
i. After each SQL statement add a statement to update the version in the
database. E.g. UPDATE DBVER SET VER=n, using the same version number n.

You should end up with a SQL script that, if run using a DB tool would bring
the previous database up to date. Now the trick is to write a simple tool
that:
j. Connects to a given database.
k. Reads the current version from the database (SELECT VER FROM DBVER).
l. Reads the migration script and looks for the /*VER=n*/ markers. It skips
the sections between /*VER=n*/ markers up to and including the current
database version.
m. It reads and runs each subsequent section, one at a time, through to the
end of the script, committing after each section.

You deploy with your application and run on install (or as appropriate):
1. The migration script.
2. The migration tool.

Repeat the process e. to i. each time you make changes to the database (next
application release etc.).

I have thousands of customers and use such a system (got the idea from a
very bright fellow, thanks Brett), which is a little more complex than the
above in that: a) I don't enter the value for n with the VER=n comments as I
created a script "compiler" which automatically inserts the numbers and also
encrypts the script, b) the migration tool decrypts the script and uses the
value in the /*VAR=n*/ lines to automatically run an UPDATE statement to
update the database version at each step, eliminating the need for the
manual version update statements after each section, c) I have other tags in
the script to do things like decide when to commit, set the version
explicitly (I have two separate databases quite similar but with slight
differences with versions that are kept in sync from time to time), or to
inform the migration tool to ignore any errors in the next statement.

Now, instead of the export metadata-diff-create SQL statements you could use
a DATABASE tool that does the diff for you and creates the migration
statements automatically, as others have suggested. Append that to your
ever-growing migration script and insert the VER and update version
statements before/after each.

Why go to this effort? Well it's really quite simple anyway - migration tool
is a no-brainer (some simple text parsing and run a given SQL statement - a
hundred lines long maybe?). Any customer database can be on any version and
you have a consistent set of migration steps to bring the databases up to
date.

Now the problem for you is that you don't have a good versioning system in
place yet. So you will need to bring the customer database, any and every
database, to a consistent state at which point the versioned migration
script starts. You may be able to start with a diff of your current DB and
the first DB you deployed to customers and ignore any errors, or use your
current "re-create" technique first if the version is not present in the
database.

Good luck!

Regards,

Jarrod Hollingworth
Complete Time Tracking
http://www.complete-time-tracking.com/