-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Closed
Description
In SPARK-24251, we found that the changes to use HadoopReadOptions accidentally switched the properties that enable stats and dictionary filters. Both are enabled by default so it is unlikely that anyone will need to turn them off and there is an easy work-around, but we should fix the properties for 1.10.1. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on 1.8.x).
Reporter: Ryan Blue / @rdblue
Assignee: Gabor Szadovszky / @gszadovszky
Related issues:
- Release Parquet Java 1.10.1 (blocks)
- Wrong configuration is passed from Hadoop config to Parquet options (is duplicated by)
- Incorrect results of not(eqNullSafe) when data read from Parquet file (is related to)
PRs and other links:
Note: This issue was originally created as PARQUET-1309. Please see the migration documentation for further details.