Skip to content

Commit 5dcb72b

Browse files
committed
HADOOP-17833. clarify how inconsistencies surface
Change-Id: I8a3272b4653fedd023d2490a77ca91d8baef1129
1 parent 3c079e8 commit 5dcb72b

File tree

1 file changed

+12
-5
lines changed

1 file changed

+12
-5
lines changed

hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdataoutputstreambuilder.md

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -197,19 +197,26 @@ Here are the custom options which the S3A Connector supports.
197197

198198
Prioritize file creation performance over safety checks for filesystem consistency.
199199

200-
This
200+
This:
201201
1. Skips the `LIST` call which makes sure a file is being created over a directory.
202202
Risk: a file is created over a directory.
203203
1. Ignores the overwrite flag.
204204
1. Never issues a `DELETE` call to delete parent directory markers.
205-
Risk: if directory markers have not already been leaned up, such as when creating the
206-
parent directory or when a least one file has been created in or under the same
207-
directory using the `create()` API or `createFile()` without this option.
208205

209206
It is possible to probe an S3A Filesystem instance for this capability through
210207
the `hasPathCapability(path, "fs.s3a.create.performance")` check.
211208

212-
Using this option is the equivalent of pressing and holding down the "Electronic Stability Control"
209+
Creating files with this option over existing directories is likely
210+
to make S3A filesystem clients behave inconsistently.
211+
212+
Operations optimized for directories (e.g. listing calls) are likely
213+
to see the directory tree not the file; operations optimized for
214+
files (`getFileStatus()`, `isFile()`) more likely to see the file.
215+
The exact form of the inconsistencies, and which operations/parameters
216+
trigger this are undefined and may change between even minor releases.
217+
218+
Using this option is the equivalent of pressing and holding down the
219+
"Electronic Stability Control"
213220
button on a rear-wheel drive car for five seconds: the safety checks are off.
214221
Things wil be faster if the driver knew what you are doing.
215222
If you don't, the fact they had held the button down will

0 commit comments

Comments
 (0)