Subject Re: [Firebird-Architect] One way to scale Firebird put it on memory virtual disk or ramfs
Author Jim Starkey
It changed substantially. We never got around to implementing "coterie"
architecture to protect against network partition, but we did implement
a "majority" commit protocol that required a majority of storage
managers to acknowledge a pre-commit before a transaction could be
considered committed, which is almost as good.

And lest anyone has missed it, I am now retired from NuoDB.


On 1/22/13 11:48 AM, Dalton Calford wrote:
>
> Jim,
>
> I have been following nuodb since the first beta. I have not looked at it
> for the past few releases due to some limitations it had at the time of
> initial testing but, that testing showed that you had spread out the
> database across a single cluster of machines - if you have a break in the
> network, the version of nuodb that I was looking at could not heal.
>
> The response I received from Barry Morris/Greg Burd at Nuodb to my
> question
> is as follows
>
> *Question*
>
> "What are the recovery methods if members of the cloud detach due to
> network fragmentation – does the whole database die or does the fragments
> of the database continue to work and then reconcile the work processed
> once
> the network is re-established?"
>
> *Answer Given*
>
> "Should a network partition occur only one segment will remain active and
> only if that segment meets the minimal requirements defined by the systems
> administrator. Other segments will remain offline until the partition is
> resolved, then they will rejoin and synchronize data with the master
> segment. If your design require a system that can partition, run
> independently and then synchronize when reconnected then we suggest that
> each logically independent segment of the database in fact be its own
> physical database."
>
> Has that answer changed in the newer releases? Can the different segments
> separate and reattach, re-applying work without loss of data that was
> committed on two separate segments? Can those segments continue to
> work independently until the connection reconnects? Does nuodb reconcile
> the work without having the various tiers/clients needing any
> understanding
> of the various elements involved? I am truly curious as we spent years on
> in house middle tier solutions that do this for us.
>
> It is not uncommon for network switches to fail, for cables to be pulled,
> for hardware to reach end of life. When dealing with financial or
> telecommunications infrastructure, you design with this in mind - as you
> can have your data spread across multiple offices, in multiple cities, in
> multiple states/provinces in multiple countries. You set up
> batches/logs/authoritative servers etc. so that the work that did not
> fully
> commit can be reapplied and so that the clients know that the data
> they are
> viewing may not be authoritative. You do not try to commit to all servers
> at all locations as latency makes this dubious at best. There are ways to
> deal with it - some as old as the original double sided accounting
> processes, but each process must be followed from the client to the server
> or you find problems throughout.
>
> From what was presented to me with nuodb, was a way of spreading out your
> processing/risk across a single pop. You would still treat each pop as a
> single server and follow other procedures to have the redundancy span
> across pops while the versions I looked at did not have the means of
> programming this within the database itself. (no procedural language
> support in the beta I last looked at).
>
> Now, if you read the whole email, I began by equating the risk of ram
> drive
> vs any other medium and stated that there is risk and loss of ACID unless
> you have taken a great deal of precautions. Attacking the use of ram
> disks with a generic attack based upon the possible loss of ACID is an
> incorrect argument given the number of other factors that can reduce or
> remove ACID. A system designer/developer/solution provider must take into
> consideration many factors and balance out the cost/benefit ratios of each
> and every risk/solution. I showed that there are many such things that
> cause risk that the average developer on this list will never encounter.
> I did say such a task was possible but that Firebird would need to mirror
> the elements in some other databases.
>
> regards
>
> Dalton
>
> On 22 January 2013 10:57, Jim Starkey jim@...
> <mailto:jim%40jimstarkey.net>> wrote:
>
> > **
> >
> >
> > Dalton, that is so much nonsense. NuoDB is an existence proof that your
> > assertion is false.
> >
> > There are lots of ways to make distributed transactions ACID without
> > queries being run "atomically upon multiple systems." I demonstrated
> > one, but stopped looking after that. The inquisitive mind may want to
> > keep looking.
> >
> > Every time some says something like "The only way to ..." innovation is
> > trampled. If I had believed that "the only way to implement consistent
> > transactions is two phase locking" MVCC, Interbase, and Firebird
> > wouldn't exist.
> >
> > Programmers have learned count "zero, one, two, infinity". There is
> > never only one way to do something. There are the ways you know and the
> > ways you haven't thought of. If you stop looking, you'll never find
> them.
> >
> > There are many people who believe that all of the good ideas have been
> > discovered. They're wrong.
> >
> > On 1/22/13 9:35 AM, Dalton Calford wrote:
> > >
> > >
> > >
> > > *The only way to become fully ACID compliant is to house multiple
> > > servers,**
> > > **in multiple nocs, with queries being run atomically upon multiple
> > > systems**
> > > **in parallel so that if one system fails while processing work, the**
> > > **secondary system can respond as it has already been doing the same
> > > work as**
> > > **the primary. * The clients and tiers need to be able to determine if
> > the
> >
> > > answer they are getting is authoritative or non-authoritative.
> They need
> > > to be able to deal with multiple responses to the same query and
> all work
> > > is queue'd locally in case the entire local work process needs to be
> > > re-applied (either immediately or delayed due to network
> latency/drops).
> > >
> > > It is a very complex subject that involves the design of the pop, the
> > > power
> > > for the pop, the full system redundancy for each cluster, with
> full drive
> > > redundancy in the sans, with the client software and middle tier
> software
> > > designed to handle a complete system failure and switch-over to a
> totally
> > > different pop in a different city/province/country if needed.
> > >
> > > Ram drives work, but I normally use them for the temp drive. The
> > > benefits of ram drives depend upon the version of firebird, the
> version
> > of
> > > the OS and the hardware parameters.
> > >
> > > If firebird is proposing going down that route, they would need to
> > > implement some new structures in Firebird, mirroring some of the
> elements
> > > that oracle and other database platforms have created.
> > >
> > > On 22 January 2013 08:51, Roman Simakov roman.simakov@...
> <mailto:roman.simakov%40red-soft.biz>
> > > > wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > 2013/1/22 Sergey Mereutsa serj@...
> <mailto:serj%40dqteam.com> >:
> > > >
> > > > > Hello Alex,
> > > > >
> > > > > and without shadow ACID rule has effect too - RAM-drive is the
> same
> > > > block device as usual HDD - until power is not lost :)
> > > >
> > > > In this case you have ACI but have no D.
> > > >
> > > >
> > > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>



[Non-text portions of this message have been removed]