On 2006.01.31 05:09, Todd S. wrote:
> I'm using the Matrix module (matrix.rb)  to help place a vertex into 
> local coordinates and then back into world coordinates.  Under certain 
> situations the transformation to and from local space corrupts the 
> data...
> 
> This matrix...
>     -0.0108226964530337  -0.224934679233174 -0.974313737622412
>       0.0468247818757306    0.973187905351297 -0.225194894880513
>     -0.998844486916645      0.0480592442327619 0.0
> 
> will invert to this...
>   64.0                               0.0 
> -0.998844486916645
>     0.0                               1.0 
> 0.0480592442327619
>   -0.974313737622412 -0.225194894880513 -2.40312135813741e-18
> 
> 
> starting with vertex: 0.015525, -0.28474399, 0.53221899
> multiplyng by the inverse matrix and then back again by the matrix 
> itself will not yield the same number as it should but instead 
> returns... 0.00555723611485535, -0.24161810434515, -0.4739174731596
> 
> So am I seeing strange behavior due to the extreamly small (e-18) number 
> in the matrix?

Without pretending to have any idea about mathematical operations, 
it is quite likely that floating-pointedness is causing the problem.
You cannot really even count on 1.8 and 2.1 - 0.3 being the same.

> If so, how can I set the precision so that I don't see this error?

BigDecimal in the stdlib is, as far as I know, lossless but of 
course somewhat slower if you are doing anything very intense.



E