Support for reading SAS7BDAT files

Working version except for compression

reorganized directory structure

Added license file for Jared Hobbs code

RLE decompression

use ndarray instead of bytes

RDC decompression

Fix byte order swapping

fix rebase errors in test_xport

Use filepath_or_buffer io function

Handle alilgnment correction

Revamped testing

Add test with unicode strings

Add minimal encoding detection

Refactor row-processing

Add missing test file

Unclobber test files

Try again to revert accidental changes to test data files

Minor changes in response to code review

Add SAS benchmarks to ASV

Stash changes before rebase

refactor following code review

Updated io and whatsnew

Updates following code review

Remove local test modifications

Minor changes following code review

Remove unwanted test data file

Mostly formatting changes following code review

Remove two unneeded files

Add __init__py

Author: kshedden

LICENSES/SAS7BDAT_LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2015 Jared Hobbs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

asv_bench/benchmarks/packers.py

@@ -318,6 +318,24 @@ def remove(self, f):
             pass
 
 
+class packers_read_sas7bdat(object):
+
+    def setup(self):
+        self.f = 'data/test1.sas7bdat'
+
+    def time_packers_read_sas7bdat(self):
+        pd.read_sas(self.f, format='sas7bdat')
+
+
+class packers_read_xport(object):
+
+    def setup(self):
+        self.f = 'data/paxraw_d_short.xpt'
+
+    def time_packers_read_xport(self):
+        pd.read_sas(self.f, format='xport')
+
+
 class packers_write_csv(object):
     goal_time = 0.2
 
@@ -854,4 +872,4 @@ def remove(self, f):
         try:
             os.remove(self.f)
         except:
-            pass
+            pass

doc/source/io.rst

@@ -4564,24 +4564,25 @@ easy conversion to and from pandas.
 
 .. _io.sas_reader:
 
-SAS Format
-----------
+SAS Formats
+-----------
 
 .. versionadded:: 0.17.0
 
-The top-level function :func:`read_sas` currently can read (but
-not write) SAS xport (.XPT) format files.  Pandas cannot currently
-handle SAS7BDAT files.
+The top-level function :func:`read_sas` can read (but not write) SAS
+`xport` (.XPT) and `SAS7BDAT` (.sas7bdat) format files.
 
-XPORT files only contain two value types: ASCII text and double
-precision numeric values.  There is no automatic type conversion to
-integers, dates, or categoricals.  By default the whole file is read
-and returned as a ``DataFrame``.
+SAS files only contain two value types: ASCII text and floating point
+values (usually 8 bytes but sometimes truncated).  For xport files,
+there is no automatic type conversion to integers, dates, or
+categoricals.  For SAS7BDAT files, the format codes may allow date
+variables to be automatically converted to dates.  By default the
+whole file is read and returned as a ``DataFrame``.
 
-Specify a ``chunksize`` or use ``iterator=True`` to obtain an
-``XportReader`` object for incrementally reading the file.  The
-``XportReader`` object also has attributes that contain additional
-information about the file and its variables.
+Specify a ``chunksize`` or use ``iterator=True`` to obtain reader
+objects (``XportReader`` or ``SAS7BDATReader``) for incrementally
+reading the file.  The reader objects also have attributes that
+contain additional information about the file and its variables.
 
 Read a SAS XPORT file:
 
@@ -4602,6 +4603,8 @@ web site.
 
 .. _specification: https://support.sas.com/techsup/technote/ts140.pdf
 
+No official documentation is available for the SAS7BDAT format.
+
 .. _io.perf:
 
 Performance Considerations

doc/source/whatsnew/v0.18.0.txt

@@ -403,6 +403,13 @@ For example, if you have a jupyter notebook you plan to convert to latex using n
 Options ``display.latex.escape`` and ``display.latex.longtable`` have also been added to the configuration and are used automatically by the ``to_latex``
 method. See the :ref:`options documentation<options>` for more info.
 
+SAS7BDAT files
+^^^^^^^^^^^^^^
+
+Pandas can now read SAS7BDAT files, including compressed files.  The
+files can be read in entirety, or incrementally.  For full details see
+:ref:`here <io.sas>`. (issue:`4052`)
+
 .. _whatsnew_0180.enhancements.other:
 
 Other enhancements

pandas/io/api.py

@@ -11,7 +11,7 @@
 from pandas.io.json import read_json
 from pandas.io.html import read_html
 from pandas.io.sql import read_sql, read_sql_table, read_sql_query
-from pandas.io.sas import read_sas
+from pandas.io.sas.sasreader import read_sas
 from pandas.io.stata import read_stata
 from pandas.io.pickle import read_pickle, to_pickle
 from pandas.io.packers import read_msgpack, to_msgpack