Subject | Re: [ib-support] Re: IB stored proc & casting issue |
---|---|
Author | Paul Schmidt |
Post date | 2002-05-31T12:10:50Z |
On 30 May 2002 at 18:21, csswa wrote:
(maybe it never worked correctly -- not a lot of applications do a lot of math on the
SQL side) so it needs some checking, personally I think when dealing with
numbers, if there is a float or double anywhere it gets cast to double, and dealt with
that way. If there is no float or double, it gets cast to INT64 and you do a modulus,
if the result is 0 do the math in INT64 cast the result into the same as the longest
input value, or the shortest value that will contain the result, whichever is longer.
If the modulus result is not 0, cast everything to doubles do the math as doubles
and if the result contains a decimal point leave it as a double, otherwise cast it into
the same as the longest input value, or the shortest value that will contain the result,
whichever is longer.
Tricat Technologies
paul@...
www.tricattechnologies.com
> (yes, I am slowly burying that subject line!)I think that possibly adding 64bit Integers broke a few things in the math routines
>
> Sure, this is a neat example, Paul. But I'd still much rather have
> divisions that work as seamlessly as multiplication. I'm still hoping
> someone will point out why multiplications get automatic float-
> casting internally but divisions don't. :-)
(maybe it never worked correctly -- not a lot of applications do a lot of math on the
SQL side) so it needs some checking, personally I think when dealing with
numbers, if there is a float or double anywhere it gets cast to double, and dealt with
that way. If there is no float or double, it gets cast to INT64 and you do a modulus,
if the result is 0 do the math in INT64 cast the result into the same as the longest
input value, or the shortest value that will contain the result, whichever is longer.
If the modulus result is not 0, cast everything to doubles do the math as doubles
and if the result contains a decimal point leave it as a double, otherwise cast it into
the same as the longest input value, or the shortest value that will contain the result,
whichever is longer.
> -- Got a new truck for my wife - not a bad trade huh?Paul Schmidt, President
>
>
> --- In ib-support@y..., "Paul Schmidt" <paul@t...> wrote:
> > On 30 May 2002 at 13:59, rrokytskyy wrote:
> >
> > > > The problem is this:
> > > >
> > > > Take the diameter of an old metal swimming pool I had in the
> 1970's
> > > 18 feet, now I
> > > > want the circumference in metres.
> > > >
> > > > 'Calc = ' || '18' * .3048 * '3.14' the output variable is a
> VARCHAR
> > > (20)
> > > >
> > > > what it really needs to do is convert the 18 to a double and
> the
> > > 3.14 to a double, do
> > > > the math and cast the result to a string, concatenate that to
> any
> > > strings and sanity
> > > > check the result for length. In this case the result would be
> Calc
> > > = 17.227296 at
> > > > least according to my Canon F-502 calculator.
> > >
> > > Do not want to comment your statement about IB, but the right
> > > statement would be:
> > >
> > > 'Calc = ' || ('18' * .3048 * '3.14')
> > >
> > > at leaset this works:
> > >
> > > select 'Calc = ' || ('18' * .3048 * '3.14') from rdb$database
> > >
> > > Usually statements are evaluated from left to right, and there is
> not
> > > precedence between * and || operators.
> > >
> >
> > Okay, so I missed a couple of brackets, I should have warned it was
> untested and
> > off the top of my head.
> >
> >
> >
> >
> > Paul Schmidt, President
> > Tricat Technologies
> > paul@t...
> > www.tricattechnologies.com
>
>
> ------------------------ Yahoo! Groups Sponsor
> ---------------------~--> Tied to your PC? Cut Loose and Stay
> connected with Yahoo! Mobile
> http://us.click.yahoo.com/QBCcSD/o1CEAA/sXBHAA/67folB/TM
> ---------------------------------------------------------------------~
> ->
>
> To unsubscribe from this group, send an email to:
> ib-support-unsubscribe@egroups.com
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
Tricat Technologies
paul@...
www.tricattechnologies.com