Comparison of float data types.

• 
  Hello,
  
  I think this is the same issue we've always had where the results in the result grid are rounded off, but the actual data (in binary form) is still different. Maybe it would be possible to see this if you selected the data from one of the columns and CAST it as bytes.

  Brian Donahue February 28, 2006 12:28

  - 0 +
• 
  That was it exactly - once I queried the fields and CAST them out further I saw the exact values.

  eroach99 March 01, 2006 08:45

  - 0 +
• 
  Hi,
  
  I am having the same problem.
  
  I was wondering if you could give me an example of how to CAST the values. Thanks

  masterbaker March 06, 2006 18:09

  - 0 +
• 
  The compare apparently rounds at 13 decimal places - in our case there were more data past 13 decimal places - we used the following to see this :
  
  select cast(column as decimal(25,15)) from table

  eroach99 March 07, 2006 08:25

  - 0 +

Add comment

Please sign in to leave a comment.