Remove astropy dependency

This removal breaks the circular dependency

astropy --> pytest-astropy --> pytest-arraydiff

which may be problematic when upgrading or so. It also eases the use
of pytest-arraydiff in non-astropy environments. Astropy is only
needed to compare FITS files.

Author: olebole

setup.cfg

@@ -32,6 +32,9 @@ setup_requires =
 install_requires =
     pytest>=4.6
     numpy
+
+[options.extras_require]
+test =
     astropy
 
 [options.entry_points]