Subject | Re: [firebird-support] Definig variable as a domain |
---|---|
Author | Randall Sell |
Post date | 2010-03-17T06:39:07Z |
Hi Helen,
I wasn't aware I was introducing any precision error. The original code actually had this comment:
/* casting here is done for best accuracy http://www.firebirdfaq.org/faq207/ */
But I took it out so as to focus on the issue. But that FAQ states that we should be casting as double precision. And we are casting as double precision. So where is the precision error creaping in?
You stated that perhaps we should be using a function. Surely FB is
capable of dividing 2 double precisions, and coming out with an
accurate result? If not, we are using the wrong DB.
You also stated that "A computed column must return a scalar". What exactly is a scalar? I thought a scalar was any numeric (x,y) definition.
cheers,
-randall
But I wouldn't hold my breath. You don't seem overly concerned about the likely precision errors you are laying into this SP. If you want a function, write a UDF.
./heLen
[Non-text portions of this message have been removed]
I wasn't aware I was introducing any precision error. The original code actually had this comment:
/* casting here is done for best accuracy http://www.firebirdfaq.org/faq207/ */
But I took it out so as to focus on the issue. But that FAQ states that we should be casting as double precision. And we are casting as double precision. So where is the precision error creaping in?
You stated that perhaps we should be using a function. Surely FB is
capable of dividing 2 double precisions, and coming out with an
accurate result? If not, we are using the wrong DB.
You also stated that "A computed column must return a scalar". What exactly is a scalar? I thought a scalar was any numeric (x,y) definition.
cheers,
-randall
But I wouldn't hold my breath. You don't seem overly concerned about the likely precision errors you are laying into this SP. If you want a function, write a UDF.
./heLen
[Non-text portions of this message have been removed]