"James Edward Gray II" <james / grayproductions.net> schrieb im Newsbeitrag
news:0F54A0B4-0723-11D9-B224-000A95BA45F8 / grayproductions.net...
> I'm running a Ruby program that relies on Bignum values.
>
> I have a correct output to compare against and test my work.  My
> current output agrees with the correct output for some time, but
> eventually, they begin to diverge.
>
> I've tried to track the problem down and it SEEMS to be that the Bignum
> values stop matching their counterparts.  (Note:  I can't actually see
> the large numbers in the correct output, so I'm guessing from behavior,
> but I'm fairly certain now.)

Err, can't you just compare them?  If it only *seems* that values differ
then there could be any number of other reasons for the differences you
observe.

> The Bignum manipulations involve / and %, so I'm guessing it's a
> precision error.  I know in Perl, I can adjust the precision of it's
> Bignum-like library.
>
> So, my question is, how do I adjust Bignum's precision for thinks like
> division?

Bignum ist just integer numbers.  AFAIK there is no such thing as a
precision for this - all digits are correct.  Since I don't know the math
you're doing it's hard to guess.  One candidate that does make a
difference with interger divisions is order of execution: you might not
execute all operations in the same order thus cutting off something which
isn't cut off in another calculation, hence they differ.

I guess you're aware what I mean:

>> 33 / 2 * 2
=> 32
>> 33 * 2 / 2
=> 33

Another source of suble errors might be multiple conversions between float
and integer numbers.

Mabye you better use Rational:

>> (Rational(33) / 2 * 2).to_i
=> 33

Or are you talking about BigDecimal?

Kind regards

    robert