Subject Re: [firebird-support] Data Partitioning with Firebird
Author Lester Caine
On 06/08/14 02:41, Nigel Weeks nigel.weeks@...
[firebird-support] wrote:
> I've been using Firebird for years, but mainly keeping a very low profile.
>
> I'm planning a Health Informatics project with the University of
> Tasmania, and as I've used Firebird to power all the other projects, I'm
> planning on using it again.
>
> The question I have is about data partitioning across servers.
> Essentially, we're wanting 10 servers to handle the backend - not
> because it can't do it with one, but we need to show that the DB can
> scale (investors love clusters).
>
> Initially, as every table is based on bigint primary keys, I could have
> server 0 handling every record ending in a zero(0,10,20,etc), with
> duplicated table structures on top.
>
> The middle layer handles the MapReduce, determining which server to ask
> for data, and combining data sets(reducing) for the app on top. (No,
> it's not BigData, just an entity mapping layer).
>
> When it comes to data partitioning across servers, what other strategies
> have people done?

I keep looking at this problem and always seem to come back to a 2 level
system. Sort of like 'telephone directory' and 'data store'. All servers
have the telephone directory, and the primary key for each entry is
essentially two 32 bit numbers creating the BIGINT value. That should
cope with just about anything? The telephone directory has all the
lookup data such as National Insurance number, passport number and the
like, and once you have the primary key then you know where to go to
pick up the rest of the data. It is a 'cloud' in the sense that every
service point will have it's own local data archive, and that can be
replicated to other data stores and the 'telephone directory' will know
if there are alternate copies of the data that can be accessed should
the primary one be off line.

I think that Firebird's backup strengths makes this something Firebird
should excel at? Replicating or simply cloning material as required?

--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk