Subject | Re: [Firebird-Architect] One way to scale Firebird put it on memory virtual disk or ramfs |
---|---|
Author | Jim Starkey |
Post date | 2013-01-22T20:15:24Z |
It changed substantially. We never got around to implementing "coterie"
architecture to protect against network partition, but we did implement
a "majority" commit protocol that required a majority of storage
managers to acknowledge a pre-commit before a transaction could be
considered committed, which is almost as good.
And lest anyone has missed it, I am now retired from NuoDB.
architecture to protect against network partition, but we did implement
a "majority" commit protocol that required a majority of storage
managers to acknowledge a pre-commit before a transaction could be
considered committed, which is almost as good.
And lest anyone has missed it, I am now retired from NuoDB.
On 1/22/13 11:48 AM, Dalton Calford wrote:
>
> Jim,
>
> I have been following nuodb since the first beta. I have not looked at it
> for the past few releases due to some limitations it had at the time of
> initial testing but, that testing showed that you had spread out the
> database across a single cluster of machines - if you have a break in the
> network, the version of nuodb that I was looking at could not heal.
>
> The response I received from Barry Morris/Greg Burd at Nuodb to my
> question
> is as follows
>
> *Question*
>
> "What are the recovery methods if members of the cloud detach due to
> network fragmentation – does the whole database die or does the fragments
> of the database continue to work and then reconcile the work processed
> once
> the network is re-established?"
>
> *Answer Given*
>
> "Should a network partition occur only one segment will remain active and
> only if that segment meets the minimal requirements defined by the systems
> administrator. Other segments will remain offline until the partition is
> resolved, then they will rejoin and synchronize data with the master
> segment. If your design require a system that can partition, run
> independently and then synchronize when reconnected then we suggest that
> each logically independent segment of the database in fact be its own
> physical database."
>
> Has that answer changed in the newer releases? Can the different segments
> separate and reattach, re-applying work without loss of data that was
> committed on two separate segments? Can those segments continue to
> work independently until the connection reconnects? Does nuodb reconcile
> the work without having the various tiers/clients needing any
> understanding
> of the various elements involved? I am truly curious as we spent years on
> in house middle tier solutions that do this for us.
>
> It is not uncommon for network switches to fail, for cables to be pulled,
> for hardware to reach end of life. When dealing with financial or
> telecommunications infrastructure, you design with this in mind - as you
> can have your data spread across multiple offices, in multiple cities, in
> multiple states/provinces in multiple countries. You set up
> batches/logs/authoritative servers etc. so that the work that did not
> fully
> commit can be reapplied and so that the clients know that the data
> they are
> viewing may not be authoritative. You do not try to commit to all servers
> at all locations as latency makes this dubious at best. There are ways to
> deal with it - some as old as the original double sided accounting
> processes, but each process must be followed from the client to the server
> or you find problems throughout.
>
> From what was presented to me with nuodb, was a way of spreading out your
> processing/risk across a single pop. You would still treat each pop as a
> single server and follow other procedures to have the redundancy span
> across pops while the versions I looked at did not have the means of
> programming this within the database itself. (no procedural language
> support in the beta I last looked at).
>
> Now, if you read the whole email, I began by equating the risk of ram
> drive
> vs any other medium and stated that there is risk and loss of ACID unless
> you have taken a great deal of precautions. Attacking the use of ram
> disks with a generic attack based upon the possible loss of ACID is an
> incorrect argument given the number of other factors that can reduce or
> remove ACID. A system designer/developer/solution provider must take into
> consideration many factors and balance out the cost/benefit ratios of each
> and every risk/solution. I showed that there are many such things that
> cause risk that the average developer on this list will never encounter.
> I did say such a task was possible but that Firebird would need to mirror
> the elements in some other databases.
>
> regards
>
> Dalton
>
> On 22 January 2013 10:57, Jim Starkey jim@...
> <mailto:jim%40jimstarkey.net>> wrote:
>
> > **
> >
> >
> > Dalton, that is so much nonsense. NuoDB is an existence proof that your
> > assertion is false.
> >
> > There are lots of ways to make distributed transactions ACID without
> > queries being run "atomically upon multiple systems." I demonstrated
> > one, but stopped looking after that. The inquisitive mind may want to
> > keep looking.
> >
> > Every time some says something like "The only way to ..." innovation is
> > trampled. If I had believed that "the only way to implement consistent
> > transactions is two phase locking" MVCC, Interbase, and Firebird
> > wouldn't exist.
> >
> > Programmers have learned count "zero, one, two, infinity". There is
> > never only one way to do something. There are the ways you know and the
> > ways you haven't thought of. If you stop looking, you'll never find
> them.
> >
> > There are many people who believe that all of the good ideas have been
> > discovered. They're wrong.
> >
> > On 1/22/13 9:35 AM, Dalton Calford wrote:
> > >
> > >
> > >
> > > *The only way to become fully ACID compliant is to house multiple
> > > servers,**
> > > **in multiple nocs, with queries being run atomically upon multiple
> > > systems**
> > > **in parallel so that if one system fails while processing work, the**
> > > **secondary system can respond as it has already been doing the same
> > > work as**
> > > **the primary. * The clients and tiers need to be able to determine if
> > the
> >
> > > answer they are getting is authoritative or non-authoritative.
> They need
> > > to be able to deal with multiple responses to the same query and
> all work
> > > is queue'd locally in case the entire local work process needs to be
> > > re-applied (either immediately or delayed due to network
> latency/drops).
> > >
> > > It is a very complex subject that involves the design of the pop, the
> > > power
> > > for the pop, the full system redundancy for each cluster, with
> full drive
> > > redundancy in the sans, with the client software and middle tier
> software
> > > designed to handle a complete system failure and switch-over to a
> totally
> > > different pop in a different city/province/country if needed.
> > >
> > > Ram drives work, but I normally use them for the temp drive. The
> > > benefits of ram drives depend upon the version of firebird, the
> version
> > of
> > > the OS and the hardware parameters.
> > >
> > > If firebird is proposing going down that route, they would need to
> > > implement some new structures in Firebird, mirroring some of the
> elements
> > > that oracle and other database platforms have created.
> > >
> > > On 22 January 2013 08:51, Roman Simakov roman.simakov@...
> <mailto:roman.simakov%40red-soft.biz>
> > > > wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > 2013/1/22 Sergey Mereutsa serj@...
> <mailto:serj%40dqteam.com> >:
> > > >
> > > > > Hello Alex,
> > > > >
> > > > > and without shadow ACID rule has effect too - RAM-drive is the
> same
> > > > block device as usual HDD - until power is not lost :)
> > > >
> > > > In this case you have ACI but have no D.
> > > >
> > > >
> > > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> > >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
>
[Non-text portions of this message have been removed]