Strange loss of precision multiplying big.Float
Strange loss of precision multiplying big.Float
If you parse a string into a big.Float like f.SetString("0.001")
, then multiply it, I'm seeing a loss of precision. If I use f.SetFloat64(0.001)
, I don't lose precision. Even doing a strconv.ParseFloat("0.001", 64)
, then calling f.SetFloat()
works.
f.SetString("0.001")
f.SetFloat64(0.001)
strconv.ParseFloat("0.001", 64)
f.SetFloat()
Full example of what I'm seeing here:
https://play.golang.org/p/_AyTHJJBUeL
Expanded from this question: https://stackoverflow.com/a/47546136/105562
1 Answer
1
The difference in output is due to imprecise representation of base 10 floating point numbers in float64
(IEEE-754 format) and the default precision and rounding of big.Float
.
float64
big.Float
See this simple code to verify:
fmt.Printf("%.30fn", 0.001)
f, ok := new(big.Float).SetString("0.001")
fmt.Println(f.Prec(), ok)
Output of the above (try it on the Go Playground):
0.001000000000000000020816681712
64 true
So what we see is that the float64
value 0.001
is not exactly 0.001
, and the default precision of big.Float
is 64.
float64
0.001
0.001
big.Float
If you increase the precision of the number you set via a string
value, you will see the same output:
string
s := "0.001"
f := new(big.Float)
f.SetPrec(100)
f.SetString(s)
fmt.Println(s)
fmt.Println(BigFloatToBigInt(f))
Now output will also be the same (try it on the Go Playground):
0.001
1000000000000000
SetPrec(128)
@TravisReeder I assume it is due to the fact that
0.001
cannot be represented precisely, and when using finite bits, the implementation rounds 0.001
to the closest number representable using the given precision. Sometimes it might mean to round up, sometimes to round down.– icza
Sep 4 '18 at 15:26
0.001
0.001
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Thanks, that works, and I had actually tried that, but I used
SetPrec(128)
and that doesn't work. Turns out 128 and 64 don't work, work but everything around it does... Any idea why that is? See results here: play.golang.org/p/KLnPY8cqKpe– Travis Reeder
Sep 4 '18 at 15:01