Make compression adaptive with V2 data pages

[pitrou]

Describe the enhancement requested

When writing a V2 data page, it seems that compression is always unconditionally enabled even when compression doesn't actually yield any benefits:

parquet-java/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnChunkPageWriteStore.java

Lines 305 to 311 in 0fea3e1

	boolean compressed = false;
	BytesInput compressedData = BytesInput.empty();
	if (data.size() > 0) {
	// TODO: decide if we compress
	compressedData = compressor.compress(data);
	compressed = true;
	}

It would be relatively easy to use a hardcoded threshold (for example 98%) above which compression is disabled, which makes reading faster.

Component(s)

Core

[alamb]

In case it helps for reference, @mapleFU made a similar change in the arrow-rs / Rust parquet reader a few weeks ago:

• Parquet: Do not compress v2 data page when compress is bad quality arrow-rs#8257

[pitrou]

Yes, and I agree that adding a tunable parameter would be even better. In Arrow C++ IPC we have:
https://github.com/apache/arrow/blob/03896451c69658105e857ae7103e5081bbaa9bd6/cpp/src/arrow/ipc/options.h#L72-L85

It could have a conservative default such as 0.02.

[mapleFU]

A config is ok to me. It also makes sense in https://github.com/apache/arrow/blob/03896451c69658105e857ae7103e5081bbaa9bd6/cpp/src/parquet/column_writer.cc#L1051