Jump to content

Maximum Value Using "real" Numbers


marke

Recommended Posts

Hi

 

When using real numbers, there is a potential of rounding errors if the "value" is to large.

The real or floating point number is stored as a mantissa and an exponent where the mantissa is the "value" part and the exponent is the "tens multiplier".

For example 123.45 has a mantissa of 12345 and an exponent of 10−2

 

The maximum value of the mantissa is just over 32,000,000 Any value above this will be truncated.

This becomes significant where you have a running total and are adding in small values and keeping an accurate total. - for example power usage where the value that you are adding in is not a whole number.

 

Best regards,

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...