Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
A different and better way to find prime numbers
#11
Quote:The modulus, or remainder, operator divides number1 by number2 (rounding floating-point numbers to integers) and returns only the remainder as result
I assume it's the same for QB - but that's pretty vague on what rounding to integer means... does it round floating-point values to an integer that is still stored in a floating-point representation, or does it round to integer and convert to an integer type? I would assume it is the latter (convert to integer type) because of the fact that 2,147,483,647 is the highest possible value - this is also the largest positive value a signed 4-byte integer (LONG) can hold.



Quote:I set up a test with the number DIMed as DOUBLE. When the input number was equal to 2,147,483,647 an overflow error was issued.
I also did a test and an overflow only occurred with the value 2,147,483,648, not 2,147,483,647.
Code:
DIM a AS DOUBLE, b AS DOUBLE, c AS DOUBLE
a = 2147483647#
b = 2147483647#
c = a MOD b
PRINT c
Interchanging 'a' or 'b' for any values less than or equal to 2,147,483,647 does not cause an overflow.
Reply


Messages In This Thread
A different and better way to find prime numbers - by DrV - 09-17-2005, 09:27 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)