Commit 153cf68
Prashant Khoje
ppc64le - Fix deviation in floating point values caused by FMA
This was exposed while testing CockroachDB.
Following tests from pkg/sql/opt/exec/execbuilder fail due to floating point precision differences caused by FMA:
- TestExecBuild/local/geospatial
- TestExecBuild/local-vec-off/geospatial
- TestExecBuild/local-spec-planning/geospatial
- TestExecBuild/fakedist/geospatial
- TestExecBuild/fakedist-vec-off/geospatial
- TestExecBuild/fakedist-metadata/geospatial
- TestExecBuild/fakedist-disk/geospatial
- TestExecBuild/fakedist-spec-planning/geospatial
With explicit casts in this patch, these failures are resolved.
References:
https://go.dev/ref/spec#Floating_point_operators
golang/go#53297
golang/go#48145
disintegration/gift#20 (comment)1 parent 740aa86 commit 153cf68
1 file changed
+2
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
| 67 | + | |
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
| 75 | + | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
| |||
0 commit comments