05-11-2005, 06:48 AM
I have hit this problem a couple of times now so I have to ask.
If you run the code, you will find that sometimes you get some bit-rot. i.e. .0000000000000001 or the like. I understand the limits of floating point numbers and so on. But similar code under VB (not using its Round function), C, and ForTran does not show this behaviour.
Also, I have played with just grabbing the fractional part of the number (after rounding), converting it to a string, performing the truncation, and then use VAL to return it to a number. As a string it is fine, once VAL gets it, the bit-rot returns.
Why in FB and Not C?
I posted this to the FBMathLib group, and I thought more insight might be gained here (let a view more eyes look at the problem).
Code:
Function Round (x as double, Digit_Count as integer = 0) as double
dim MaxRoundPlaces as integer = 16
dim ReturnAnswer as double = 0
If (Digit_Count >= 0) and (Digit_Count < MaxRoundPlaces) then
Dim Dec_Place as double = 10 ^ Digit_Count
ReturnAnswer = int((abs (x) * Dec_Place) + 0.5)
ReturnAnswer = Sgn(x) * ReturnAnswer / Dec_Place
else
'raise an error
end if
Round = ReturnAnswer
End Function
const c as double = 2^(1/12)
'start calculating note freq from A(0)
dim n as double = 27.5
'Find the next 4 octaves
'notice the error in the 60's
for i = 1 to 48
n *= c
? n, Round(n,4)
?
next i
sleepIf you run the code, you will find that sometimes you get some bit-rot. i.e. .0000000000000001 or the like. I understand the limits of floating point numbers and so on. But similar code under VB (not using its Round function), C, and ForTran does not show this behaviour.
Also, I have played with just grabbing the fractional part of the number (after rounding), converting it to a string, performing the truncation, and then use VAL to return it to a number. As a string it is fine, once VAL gets it, the bit-rot returns.
Why in FB and Not C?
I posted this to the FBMathLib group, and I thought more insight might be gained here (let a view more eyes look at the problem).