TestSnappy() throws OOM exception with Parquet-1485 change

[asfimport]

Parquet-1485 initialize the buffer size(inputBuffer and outputBuffer) from 0 to 128M in total. This cause the unit test TestSnappy() failed with OOM exception. This is on my Mac laptop. 

To solve the unit test failure, we can increase the size of -Xmx from 512m to 1024m like below. However, we need to evaluate whether or not the increase of the initial size of direct memory usage for inputBuffer and outputBuffer will cause real Parquet application OOM or not, if that application is not with big enough -Xmx size. 

org.apache.maven.plugins
maven-surefire-plugin
...
-Xmx1014m

...

For details of the exception, the commit page (7dcdcdc) has the detail. 

Environment: Mac OS 10.14.1
Reporter: Xinli Shang / @shangxinli
Assignee: Gabor Szadovszky / @gszadovszky

Related issues:

• Snappy Decompressor/Compressor may cause direct memory leak (relates to)

PRs and other links:

• GitHub Pull Request #622

_{Note: This issue was originally created as PARQUET-1533. Please see the migration documentation for further details.}

[asfimport]

Gabor Szadovszky / @gszadovszky:
Thanks a lot, [~shangx@uber.com] for creating this JIRA. I haven't realized during the review that we use such large default buffer sizes.

@liupc, is there a reason behind the default 64M buffer size? (Parquet uses compression on pages which are usually ~1M size.) The original implementation did not use any default size but simply initialized the buffers with 0 and let them grow. Would you like to handle this issue?