Jan 12, 2015; 14:18
Derek Caswell
math_div precision
I am having an issue that I hope someone can help with. I have these 2
lines of code:
local('discountAmount') decimal(math_mult(math_div(math_sub(#couponAmount,
#segmentFees->find(#SegmentNum)), #segmentFare->find(#SegmentNum))),
1.0000);
#discountAmount->SetFormat(-Precision=4, -GroupChar=',');
In my test case:
#couponAmount = 300
#segmentFees->find(#SegmentNum) = 37
#segmentFare->find(#SegmentNum) = 290
What I am getting is #discountAmount = 0.9100
When I do the math with a calculator, I get 0.90689655172.
I shouldn't need to go out any further then the 4th decimal place but as
you can see it just keeps rounding it to the 2nd decimal place.
We have set the global precision using Decimal_SetGlobalDefaultPrecision to
2 which I can't change. Any Ideas on how to get this to give me 0.9069
instead of 0.91?
Thanks,
Derek Caswell
Developer
SkyVantage Corporation
5526 West 13400 South, Suite 207
Herriman, UT 84096
United States of America
Email: dcaswell@skyvantage.com <crobin@skyvantage.com>
Website: http://www.skyvantage.com
Join us on Facebook! www.facebook.com/skyvantage
#############################################################
This message is sent to you because you are subscribed to
the mailing list Lasso Lasso@lists.lassosoft.com
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <Lasso-unsubscribe@lists.lassosoft.com>
Send administrative queries to <Lasso-request@lists.lassosoft.com>
Jan 12, 2015; 13:47
Steve Piercy - Website Builder
Re: math_div precision