Commit ac08a25
authored
HADOOP-11867. Add a high-performance vectored read API. (#3904)
part of HADOOP-18103.
Add support for multiple ranged vectored read api in PositionedReadable.
The default iterates through the ranges to read each synchronously,
but the intent is that FSDataInputStream subclasses can make more
efficient readers especially in object stores implementation.
Also added implementation in S3A where smaller ranges are merged and
sliced byte buffers are returned to the readers. All the merged ranged are
fetched from S3 asynchronously.
Contributed By: Owen O'Malley and Mukund Thakur1 parent d8ab842 commit ac08a25
File tree
30 files changed
+2422
-67
lines changed- dev-support
- hadoop-common-project
- hadoop-common/src
- main/java/org/apache/hadoop/fs
- impl
- site/markdown/filesystem
- test/java/org/apache/hadoop
- fs
- contract
- localfs
- rawlocal
- impl
- test
- hadoop-project
- hadoop-tools
- hadoop-aws/src
- main/java/org/apache/hadoop/fs/s3a
- test
- java/org/apache/hadoop/fs
- contract/s3a
- s3a
- resources
- hadoop-benchmark
- src/main
- assembly
- findbugs
- java/org/apache/hadoop/benchmark
30 files changed
+2422
-67
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
| 50 | + | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| |||
Lines changed: 23 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
25 | 28 | | |
26 | 29 | | |
27 | 30 | | |
| |||
158 | 161 | | |
159 | 162 | | |
160 | 163 | | |
161 | | - | |
162 | | - | |
163 | | - | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
164 | 183 | | |
165 | 184 | | |
0 commit comments