08-17-2003, 05:12 AM
In the last couple of days in the challenge regarding "useful programs", Oracle posted the following function to perform rounding to the nearest whole number:(I took the liberty of streamlining it for the general case.)
I recognized immediately that this was the most precise rounding logic that I had ever seen, because usng the ABS it rounds correctly without being burdened by the sign of the input number. Then, it restores the sign by using the SGN.
After testing Oracle's logic thoroughly, I realized that for at least the last 5 years, I have been rounding negative numbers incorrectly. :oops:
BTW: Ak00ma came up with the following enhancement of Oracle's logic, which handles numbers with implied decimal places:
Without much ado, I'll list 3 erroneous rounding methods:
1) Using the CINT. This looks very simple when your numbers to be rounded are in the range of integer values. However, if you round .5 or -.5 you get zero for both. Who knows why, and what other inputs will also fail?
2) Testing the sign and rounding by +.5 or -.5 accordingly. This was my method, but testing shows that it doesn't always work for all negative inputs. The root of the problem is stated in the QB manual: "The INT returns the largest interger less than or equal to to the input¨. Less than/equal does not work the same for positive and negative numbers. Another Basic weirdness.
3) The cavalier method. Just do: INT(NUMB+.5) and hope for the best. Works great for positive numbers, but suffers from obvious problems on negative numbers, compounded by the above less than/equal weirdness.
SUMMARY: Don't take my word for it, do your own testing. You will certainly come to realize that Oracle's approach is the only correct way to round.
BTW: Tested the INT on a negative number in Visual Basic, and got the same error. INT(-123.4) gave -124. :rotfl:
*****
_________________
Code:
DEF FNround# (m#) = SGN(m#) * INT(ABS(m#) + .5)
I recognized immediately that this was the most precise rounding logic that I had ever seen, because usng the ABS it rounds correctly without being burdened by the sign of the input number. Then, it restores the sign by using the SGN.
After testing Oracle's logic thoroughly, I realized that for at least the last 5 years, I have been rounding negative numbers incorrectly. :oops:
BTW: Ak00ma came up with the following enhancement of Oracle's logic, which handles numbers with implied decimal places:
Code:
DEF FNround# (m#, dec%) = SGN(m#) * INT(10^dec% * ABS(m#) + .5) / 10 ^ dec%
Without much ado, I'll list 3 erroneous rounding methods:
1) Using the CINT. This looks very simple when your numbers to be rounded are in the range of integer values. However, if you round .5 or -.5 you get zero for both. Who knows why, and what other inputs will also fail?
2) Testing the sign and rounding by +.5 or -.5 accordingly. This was my method, but testing shows that it doesn't always work for all negative inputs. The root of the problem is stated in the QB manual: "The INT returns the largest interger less than or equal to to the input¨. Less than/equal does not work the same for positive and negative numbers. Another Basic weirdness.
3) The cavalier method. Just do: INT(NUMB+.5) and hope for the best. Works great for positive numbers, but suffers from obvious problems on negative numbers, compounded by the above less than/equal weirdness.
SUMMARY: Don't take my word for it, do your own testing. You will certainly come to realize that Oracle's approach is the only correct way to round.
BTW: Tested the INT on a negative number in Visual Basic, and got the same error. INT(-123.4) gave -124. :rotfl:
*****
_________________