Skip to content
This repository was archived by the owner on Feb 12, 2024. It is now read-only.

Commit 1760b89

Browse files
authored
refactor: make add only work on single items (#3167)
`ipfs.add` now returns single items only. Where a source would result in multiple items returned, only the last item is returned. As a first pass `ipfs.add` still takes multiple items but only accepting single items is documented and considered supported. In the future it may throw if multiple items are passed. `ipfs.addAll` retains the previous behaviour of `ipfs.add`. Examples: ```javascript const { cid, path, mode, mtime } = await ipfs.add('Hello world') ``` ```javascript const { cid } = await ipfs.add(Uint8Array.from([0, 1, 2])) ``` ```javascript const { cid } = await ipfs.add(fs.createReadStream('/path/to/file.txt')) ``` ```javascript const { cid } = await ipfs.add({ path: '/foo/bar/baz.txt', content: 'File content' }) // Creates a DAG with multiple levels of directories. // The returned cid is the CID for the root directory /foo // You can retrieve the file content with an IPFS path for await (const buf of ipfs.cat(`/ipfs/${cid}/bar/baz.txt`)) { ... } // Or get the CID of the nested file with ipfs.files.stat const { cid: fileCid } = await ipfs.files.stat(`/ipfs/${cid}/bar/baz.txt`) // or ipfs.dag.resolve const { cid: fileCid } = await ipfs.dag.resolve(`/ipfs/${cid}/bar/baz.txt`) ``` ```javascript // To have `/foo` included in the ipfs path, wrap it in a directory: const { cid } = await ipfs.add({ path: '/foo/bar/baz.txt', content: 'File content' }, { wrapWithDirectory: true }) for await (const buf of ipfs.cat(`/ipfs/${cid}/foo/bar/baz.txt`)) { ... } ``` BREAKING CHANGES: - `ipfs.add` only works on single items - a Uint8Array, a String, an AsyncIterable<Uint8Array> etc - `ipfs.addAll` works on multiple items
1 parent 8c122e8 commit 1760b89

File tree

76 files changed

+979
-704
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+979
-704
lines changed

docs/core-api/FILES.md

Lines changed: 169 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -9,92 +9,96 @@ _Explore the Mutable File System through interactive coding challenges in our [P
99
- [Parameters](#parameters)
1010
- [Options](#options)
1111
- [Returns](#returns)
12+
- [`ipfs.addAll(source, [options])`](#ipfsaddallsource-options)
13+
- [Parameters](#parameters-1)
14+
- [Options](#options-1)
15+
- [Returns](#returns-1)
1216
- [Example](#example)
1317
- [Notes](#notes)
1418
- [Chunking options](#chunking-options)
1519
- [Hash algorithms](#hash-algorithms)
1620
- [Importing files from the file system](#importing-files-from-the-file-system)
1721
- [Importing a file from a URL](#importing-a-file-from-a-url)
1822
- [`ipfs.cat(ipfsPath, [options])`](#ipfscatipfspath-options)
19-
- [Parameters](#parameters-1)
20-
- [Options](#options-1)
21-
- [Returns](#returns-1)
22-
- [Example](#example-1)
23-
- [`ipfs.get(ipfsPath, [options])`](#ipfsgetipfspath-options)
2423
- [Parameters](#parameters-2)
2524
- [Options](#options-2)
2625
- [Returns](#returns-2)
27-
- [Example](#example-2)
28-
- [`ipfs.ls(ipfsPath)`](#ipfslsipfspath)
26+
- [Example](#example-1)
27+
- [`ipfs.get(ipfsPath, [options])`](#ipfsgetipfspath-options)
2928
- [Parameters](#parameters-3)
3029
- [Options](#options-3)
3130
- [Returns](#returns-3)
32-
- [Example](#example-3)
33-
- [The Mutable Files API](#the-mutable-files-api)
34-
- [`ipfs.files.chmod(path, mode, [options])`](#ipfsfileschmodpath-mode-options)
31+
- [Example](#example-2)
32+
- [`ipfs.ls(ipfsPath)`](#ipfslsipfspath)
3533
- [Parameters](#parameters-4)
3634
- [Options](#options-4)
3735
- [Returns](#returns-4)
38-
- [Example](#example-4)
39-
- [`ipfs.files.cp(...from, to, [options])`](#ipfsfilescpfrom-to-options)
36+
- [Example](#example-3)
37+
- [The Mutable Files API](#the-mutable-files-api)
38+
- [`ipfs.files.chmod(path, mode, [options])`](#ipfsfileschmodpath-mode-options)
4039
- [Parameters](#parameters-5)
4140
- [Options](#options-5)
4241
- [Returns](#returns-5)
43-
- [Example](#example-5)
44-
- [Notes](#notes-1)
45-
- [`ipfs.files.mkdir(path, [options])`](#ipfsfilesmkdirpath-options)
42+
- [Example](#example-4)
43+
- [`ipfs.files.cp(...from, to, [options])`](#ipfsfilescpfrom-to-options)
4644
- [Parameters](#parameters-6)
4745
- [Options](#options-6)
4846
- [Returns](#returns-6)
49-
- [Example](#example-6)
50-
- [`ipfs.files.stat(path, [options])`](#ipfsfilesstatpath-options)
47+
- [Example](#example-5)
48+
- [Notes](#notes-1)
49+
- [`ipfs.files.mkdir(path, [options])`](#ipfsfilesmkdirpath-options)
5150
- [Parameters](#parameters-7)
5251
- [Options](#options-7)
5352
- [Returns](#returns-7)
54-
- [Example](#example-7)
55-
- [`ipfs.files.touch(path, [options])`](#ipfsfilestouchpath-options)
53+
- [Example](#example-6)
54+
- [`ipfs.files.stat(path, [options])`](#ipfsfilesstatpath-options)
5655
- [Parameters](#parameters-8)
5756
- [Options](#options-8)
5857
- [Returns](#returns-8)
59-
- [Example](#example-8)
60-
- [`ipfs.files.rm(...paths, [options])`](#ipfsfilesrmpaths-options)
58+
- [Example](#example-7)
59+
- [`ipfs.files.touch(path, [options])`](#ipfsfilestouchpath-options)
6160
- [Parameters](#parameters-9)
6261
- [Options](#options-9)
6362
- [Returns](#returns-9)
64-
- [Example](#example-9)
65-
- [`ipfs.files.read(path, [options])`](#ipfsfilesreadpath-options)
63+
- [Example](#example-8)
64+
- [`ipfs.files.rm(...paths, [options])`](#ipfsfilesrmpaths-options)
6665
- [Parameters](#parameters-10)
6766
- [Options](#options-10)
6867
- [Returns](#returns-10)
69-
- [Example](#example-10)
70-
- [`ipfs.files.write(path, content, [options])`](#ipfsfileswritepath-content-options)
68+
- [Example](#example-9)
69+
- [`ipfs.files.read(path, [options])`](#ipfsfilesreadpath-options)
7170
- [Parameters](#parameters-11)
7271
- [Options](#options-11)
7372
- [Returns](#returns-11)
74-
- [Example](#example-11)
75-
- [`ipfs.files.mv(...from, to, [options])`](#ipfsfilesmvfrom-to-options)
73+
- [Example](#example-10)
74+
- [`ipfs.files.write(path, content, [options])`](#ipfsfileswritepath-content-options)
7675
- [Parameters](#parameters-12)
7776
- [Options](#options-12)
7877
- [Returns](#returns-12)
79-
- [Example](#example-12)
80-
- [Notes](#notes-2)
81-
- [`ipfs.files.flush(path, [options])`](#ipfsfilesflushpath-options)
78+
- [Example](#example-11)
79+
- [`ipfs.files.mv(...from, to, [options])`](#ipfsfilesmvfrom-to-options)
8280
- [Parameters](#parameters-13)
8381
- [Options](#options-13)
8482
- [Returns](#returns-13)
85-
- [Example](#example-13)
86-
- [`ipfs.files.ls(path, [options])`](#ipfsfileslspath-options)
83+
- [Example](#example-12)
84+
- [Notes](#notes-2)
85+
- [`ipfs.files.flush(path, [options])`](#ipfsfilesflushpath-options)
8786
- [Parameters](#parameters-14)
8887
- [Options](#options-14)
8988
- [Returns](#returns-14)
89+
- [Example](#example-13)
90+
- [`ipfs.files.ls(path, [options])`](#ipfsfileslspath-options)
91+
- [Parameters](#parameters-15)
92+
- [Options](#options-15)
93+
- [Returns](#returns-15)
9094
- [Example](#example-14)
9195

9296
## The Regular API
9397
The regular, top-level API for add, cat, get and ls Files on IPFS
9498

9599
### `ipfs.add(data, [options])`
96100

97-
> Import files and data into IPFS.
101+
> Import a file or data into IPFS.
98102
99103
#### Parameters
100104

@@ -104,19 +108,132 @@ The regular, top-level API for add, cat, get and ls Files on IPFS
104108

105109
`data` may be:
106110

107-
* `Bytes` (alias for `Buffer`|`ArrayBuffer`|`TypedArray`) [single file]
108-
* `Bloby` (alias for: `Blob`|`File`) [single file]
109-
* `string` [single file]
110-
* `FileObject` (see below for definition) [single file]
111-
* `Iterable<number>` [single file]
112-
* `Iterable<Bytes>` [single file]
113-
* `Iterable<Bloby>` [multiple files]
114-
* `Iterable<string>` [multiple files]
115-
* `Iterable<FileObject>` [multiple files]
116-
* `AsyncIterable<Bytes>` [single file]
117-
* `AsyncIterable<Bloby>` [multiple files]
118-
* `AsyncIterable<String>` [multiple files]
119-
* `AsyncIterable<FileObject>` [multiple files]
111+
* `Blob`
112+
* `String`
113+
* `Uint8Array`
114+
* `FileObject` (see below for definition)
115+
* `Iterable<Uint8Array>`
116+
* `AsyncIterable<Uint8Array>`
117+
118+
`FileObject` is a plain JS object of the following form:
119+
120+
```js
121+
{
122+
// The path you want to the file to be accessible at from the root CID _after_ it has been added
123+
path?: string
124+
// The contents of the file (see below for definition)
125+
content?: FileContent
126+
// File mode to store the entry with (see https://en.wikipedia.org/wiki/File_system_permissions#Numeric_notation)
127+
mode?: number | string
128+
// The modification time of the entry (see below for definition)
129+
mtime?: UnixTime
130+
}
131+
```
132+
133+
If no `path` is specified, then the item will be added to the root level and will be given a name according to it's CID.
134+
135+
If no `content` is passed, then the item is treated as an empty directory.
136+
137+
One of `path` or `content` _must_ be passed.
138+
139+
`FileContent` is one of the following types:
140+
141+
```js
142+
Uint8Array | Blob | String | Iterable<Uint8Array> | AsyncIterable<Uint8Array>
143+
```
144+
145+
`UnixTime` is one of the following types:
146+
147+
```js
148+
Date | { secs: number, nsecs?: number } | number[]
149+
```
150+
151+
As an object, `secs` is the number of seconds since (positive) or before (negative) the Unix Epoch began and `nsecs` is the number of nanoseconds since the last full second.
152+
153+
As an array of numbers, it must have two elements, as per the output of [`process.hrtime()`](https://nodejs.org/dist/latest/docs/api/process.html#process_process_hrtime_time).
154+
155+
#### Options
156+
157+
An optional object which may have the following keys:
158+
159+
| Name | Type | Default | Description |
160+
| ---- | ---- | ------- | ----------- |
161+
| chunker | `String` | `'size-262144` | chunking algorithm used to build ipfs DAGs |
162+
| cidVersion | `Number` | `0` | the CID version to use when storing the data |
163+
| hashAlg | `String` | `'sha2-256'` | multihash hashing algorithm to use |
164+
| onlyHash | `boolean` | `false` | If true, will not add blocks to the blockstore |
165+
| pin | `boolean` | `true` | pin this object when adding |
166+
| progress | function | `undefined` | a function that will be called with the byte length of chunks as a file is added to ipfs |
167+
| rawLeaves | `boolean` | `false` | if true, DAG leaves will contain raw file data and not be wrapped in a protobuf |
168+
| trickle | `boolean` | `false` | if true will use the [trickle DAG](https://godoc.org/github.com/ipsn/go-ipfs/gxlibs/github.com/ipfs/go-unixfs/importer/trickle) format for DAG generation |
169+
| wrapWithDirectory | `boolean` | `false` | Adds a wrapping node around the content |
170+
| timeout | `Number` | `undefined` | A timeout in ms |
171+
| signal | [AbortSignal][] | `undefined` | Can be used to cancel any long running requests started as a result of this call |
172+
173+
#### Returns
174+
175+
| Type | Description |
176+
| -------- | -------- |
177+
| `UnixFSEntry` | A object describing the added data |
178+
179+
Each yielded object is of the form:
180+
181+
```JavaScript
182+
{
183+
path: '/tmp/myfile.txt',
184+
cid: CID('QmHash'),
185+
mode: Number,
186+
mtime: { secs: Number, nsecs: Number },
187+
size: 123
188+
}
189+
```
190+
191+
#### Example
192+
193+
```js
194+
const file = {
195+
path: '/tmp/myfile.txt',
196+
content: 'ABC'
197+
}
198+
199+
const result of await ipfs.add(content)
200+
201+
console.info(result)
202+
203+
/*
204+
Prints:
205+
{
206+
"path": "tmp",
207+
"cid": CID("QmWXdjNC362aPDtwHPUE9o2VMqPeNeCQuTBTv1NsKtwypg"),
208+
"mode": 493,
209+
"mtime": { secs: Number, nsecs: Number },
210+
"size": 67
211+
}
212+
*/
213+
```
214+
215+
Now [ipfs.io/ipfs/Qm..pg/myfile.txt](https://ipfs.io/ipfs/QmWXdjNC362aPDtwHPUE9o2VMqPeNeCQuTBTv1NsKtwypg/myfile.txt) returns the "ABC" string.
216+
217+
### `ipfs.addAll(source, [options])`
218+
219+
> Import multiple files and data into IPFS.
220+
221+
#### Parameters
222+
223+
| Name | Type | Description |
224+
| ---- | ---- | ----------- |
225+
| source | Object | Data to import (see below) |
226+
227+
`source` may be:
228+
229+
* `Iterable<Blob>`
230+
* `Iterable<String>`
231+
* `Iterable<Uint8Array>`
232+
* `Iterable<FileObject>`
233+
* `AsyncIterable<Blob>`
234+
* `AsyncIterable<String>`
235+
* `AsyncIterable<Uint8Array>`
236+
* `AsyncIterable<FileObject>`
120237

121238
`FileObject` is a plain JS object of the following form:
122239

@@ -142,7 +259,7 @@ One of `path` or `content` _must_ be passed.
142259
`FileContent` is one of the following types:
143260

144261
```js
145-
Bytes | Bloby | string | Iterable<number> | Iterable<Bytes> | AsyncIterable<Bytes>
262+
Uint8Array | Blob | String | Iterable<Uint8Array> | AsyncIterable<Uint8Array>
146263
```
147264

148265
`UnixTime` is one of the following types:
@@ -179,7 +296,7 @@ An optional object which may have the following keys:
179296

180297
| Type | Description |
181298
| -------- | -------- |
182-
| `AsyncIterable<Object>` | An async iterable that yields objects describing the added data |
299+
| `AsyncIterable<UnixFSEntry>` | An async iterable that yields objects describing the added data |
183300

184301
Each yielded object is of the form:
185302

@@ -201,7 +318,7 @@ const files = [{
201318
content: 'ABC'
202319
}]
203320

204-
for await (const result of ipfs.add(content)) {
321+
for await (const result of ipfs.addAll(content)) {
205322
console.log(result)
206323
}
207324

@@ -266,7 +383,7 @@ const addOptions = {
266383
timeout: 10000
267384
};
268385

269-
for await (const file of ipfs.add(globSource('./docs', globSourceOptions), addOptions)) {
386+
for await (const file of ipfs.addAll(globSource('./docs', globSourceOptions), addOptions)) {
270387
console.log(file)
271388
}
272389

@@ -295,9 +412,8 @@ const { urlSource } = IPFS
295412

296413
const ipfs = await IPFS.create()
297414

298-
for await (const file of ipfs.add(urlSource('https://ipfs.io/images/ipfs-logo.svg'))) {
299-
console.log(file)
300-
}
415+
const file = await ipfs.add(urlSource('https://ipfs.io/images/ipfs-logo.svg'))
416+
console.log(file)
301417

302418
/*
303419
{

examples/browser-add-readable-stream/index.js

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,13 +52,13 @@ const streamFiles = async (ipfs, directory, files) => {
5252
}
5353
})
5454

55-
for await (const data of ipfs.add(stream)) {
56-
log(`Added ${data.path} hash: ${data.hash}`)
55+
const data = await ipfs.add(stream)
5756

58-
// The last data event will contain the directory hash
59-
if (data.path === directory) {
60-
return data.cid
61-
}
57+
log(`Added ${data.path} hash: ${data.hash}`)
58+
59+
// The last data event will contain the directory hash
60+
if (data.path === directory) {
61+
return data.cid
6262
}
6363
}
6464

examples/browser-browserify/src/index.js

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,11 @@ document.addEventListener('DOMContentLoaded', async () => {
1010
async function store () {
1111
const toStore = document.getElementById('source').value
1212

13-
for await (const file of node.add(toStore)) {
14-
if (file && file.cid) {
15-
console.log('successfully stored', file.cid)
13+
const file = await node.add(toStore)
1614

17-
await display(file.cid)
18-
}
19-
}
15+
console.log('successfully stored', file.cid)
16+
17+
await display(file.cid)
2018
}
2119

2220
async function display (cid) {

examples/browser-parceljs/public/index.js

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,18 +22,17 @@ document.addEventListener('DOMContentLoaded', async () => {
2222

2323
log(`The IPFS node version is ${version.version}`)
2424

25-
for await (const entry of node.add({
25+
const entry = await node.add({
2626
path: 'hello-parcel.txt',
2727
content: 'Hello from parcel.js bundled ipfs example'
28-
})) {
29-
log(`This page deployed ${entry.path} to IPFS and its CID is ${entry.cid}`)
28+
})
29+
log(`This page deployed ${entry.path} to IPFS and its CID is ${entry.cid}`)
3030

31-
const buffers = []
31+
const buffers = []
3232

33-
for await (const buf of node.cat(entry.cid)) {
34-
buffers.push(buf)
35-
}
36-
37-
log(`The contents of the file was: ${Buffer.concat(buffers).toString()}`)
33+
for await (const buf of node.cat(entry.cid)) {
34+
buffers.push(buf)
3835
}
36+
37+
log(`The contents of the file was: ${Buffer.concat(buffers).toString()}`)
3938
})

examples/browser-readablestream/utils.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ const dragDrop = (ipfs) => {
4545

4646
const progress = log(`IPFS: Adding...`)
4747

48-
for await (const added of ipfs.add(files, {
48+
for await (const added of ipfs.addAll(files, {
4949
progress: (addedBytes) => {
5050
progress.textContent = `IPFS: Adding ${addedBytes} bytes\r\n`
5151
}

0 commit comments

Comments
 (0)