08-09-2004, 02:45 PM
actually at 2bits I might be beat also. four works
computer adds .1 9 times and comes up with .8000001
|
08-09-2004, 02:45 PM
actually at 2bits I might be beat also. four works
08-09-2004, 02:58 PM
Just because it views in 32bits doesn't meen it can't handle a 64 bit problem. It just meens you have to "teach it".
08-09-2004, 04:44 PM
There are two solutions to this problem; both are very easy to implement. The first is to round your answer to the number of decimal points that you want:
Code: CLS The second solution, which has already been posted by Nex, is to use fixed-point: Code: CONST DIV = 1000 |
« Next Oldest | Next Newest »
|