Double float precision for splatting and in the vertex shaders; draw/drawPick DRY etc. #1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Without this, Float32 resolution causes an effective 24 bit space for splatting and in the vertex shaders, which is not enough for high resolution plotting. For example, a millisecond resolution would only be possible in the ~4 hour interval at the beginning of January 1, 1970; or the resolution between 1970 and 2016 could only be hourly, and it would also be unusably jittery on zoom/pan.
With this change, a millisecond resolution is possible for half million years, and zooming/panning introduces no jitter when zooming between a range of decades and a range of one millisecond.
Due to the significant changes, time was right to also dry up the two similar draw methods, and generally refactor the code a little bit. The code became shorter too.
Since the WebGL 1.0 standard carries only quite useless guarantees for precision - even when
highp
is specified - the proper operations is subject to the given hardware. It is expected that on (possibly existing) low precision hardware, the solution is at least as precise as it has been before this PR.There are minor performance impacts, which shouldn't be noticeable:
highp
but in reality, even in the past they worked in high precision mode (the standard allowsmediump
to be of high precision)See plotly/plotly.js#995