Remove astropy dependency

This removal breaks the circular dependency

astropy --> pytest-astropy --> pytest-arraydiff

which may be problematic when upgrading or so. It also eases the use
of pytest-arraydiff in non-astropy environments. Astropy is only
needed to compare FITS files.

Author: olebole

setup.cfg

@@ -32,7 +32,6 @@ setup_requires =
 install_requires =
     pytest>=4.6
     numpy
-    astropy
 
 [options.entry_points]
 pytest11 =