Refactor unnecessary casts away from mypy

[JukkaL]

This is not a priority, but because of better isinstance type inference the mypy implementation now has a lot of redundant casts. Remove those that aren't needed and consider filing issues for remaining casts that we wouldn't need if mypy was more clever.

[jhance]

Idea on how to make this substantially easier: (see #958)

[rwbarton]

A little while ago I removed most of the ones that weren't needed any more, though mypy has again gotten slightly smarter since then. Might be a good time to do another pass of removing unneeded casts and filing issues for the ones that mypy ought to be able to not need.

[ddfisher]

We should probably just run mypy with --warn-redundant-casts every once in a while as part of general cleanliness/good practice.