Slider pulls focus back to itself when I try to manually move it elsewhere onChangeEnd

[snowystinger]

Discussed in #4550

https://codesandbox.io/s/spring-glitter-329tzn?file=/src/App.js

1. Using the mouse, move the slider
2. Press Esc

See in the console that the button briefly receives focus before it loses it back to the thumb, which displays the focus ring after hitting Esc.