IPFS integration

[daviddias]

Hi @pfraze, it was great to chat at the #DWebSummit and getting to know about beaker.

Opening this issue so that we can continue our convo about IPFS integration and also invite other people in the community that might want to participate :)

Since beaker is an electron app, we can integrate either go-ipfs or js-ipfs, which in the short term, will give beaker's IPFS integration different properties, namely:

• Peer Routing + Content Routing (DHT) with go-ipfs
• WebRTC with js-ipfs.

Either way, we are standardizing the js-ipfs-api (HTTP-API client library) and js-ipfs core APi to expose the same calls, so that devs don't have to change any code when using a remote daemon or a in process daemon.

I still need to look into beaker code to get familiar with it, but meanwhile if you can give some lights on the integration should be done, it would be great.

Meanwhile, here is a short list of things that we will be doing in our side:

• upgrade ipfsd to use bin-wrapper Upgrade js-ipfsd-ctl to use bin-wrapper ipfs/js-ipfsd-ctl#78 so that it can be added to any electron app independently of the architecture.
• finalize interface-ipfs-core https://github.com/ipfs/interface-ipfs-core

[pfrazee]

@diasdavid great, thanks for opening this issue.

This document explains the repo. I think the background process is the right place to integrate the transport protocols, so that data can persist across navigations, and so we're not confined to WebRTC. So, go-ipfs seems like the right call.

Most of my dat integration's structure should be copyable for IPFS. There may be ways to reuse code btwn the protocol integrations, which will also pay off when we integrate webtorrent. No need to focus on that yet, though; let's just make it work, first.

After we have an ipfs:// and view-ipfs:// protocol, we can look into injecting the IPFS API into web pages.

Some info for reference:

• npm-go-ipfs ships prebuilt go-ipfs binaries, so devs should not need go installed to run beaker's build
• When Upgrade js-ipfsd-ctl to use bin-wrapper ipfs/js-ipfsd-ctl#78 closes, js-ipfsd-ctl will install the arch-appropriate ipfs binary on first execution. We'll need to hook into this and tell the user what's going on.
• js-ipfs-api is how we'll interact with the go binary.

[pfrazee]

@diasdavid @jbenet this is what I have in mind for integration right now. I shared it with the dat guys and they think it's doable on their end. I'm interested if you have technical or design issues to raise.

[pfrazee]

I've hit an issue that'll need some thought.

Chromium (and thus Electron) have a concept of "standard" URLs. You can read about them here.

To make electron's webviews handle paths correctly (in links, in imgs, etc) we want Electron to treat the folder hash as the domain. Then, paths like /icon.png will resolve as expected:

in ipfs://QmTqDSxMH6xsKabqhBSeBVu89UmbF2Z1CbN4k4tT6Zhy8R/, it'll resolve to...
ipfs://QmTqDSxMH6xsKabqhBSeBVu89UmbF2Z1CbN4k4tT6Zhy8R/icon.png

Here's the problem: the domain in standard URIs are case-insensitive. Once a protocol is registered as standard, chromium automatically runs toLowerCase on the domain. In base58/64, this destroys information.

I've committed a basic integration. If you serve an index.html via ipfs, then remove 'ipfs' from this line, you can open the file via ipfs://{root-hash-that-includes-the-index.html}. But, if you readd 'ipfs' to the standard protocol registrations, you'll see the url bar automatically toLowerCase(), and the GET will fail.

Two possible solutions:

1. Only use hex-encoded IPFS urls. This may not be super feasible, since the rest of the ecosystem is in base52+.
2. Dont use chromium's standard-url protocols. This may work, but it won't be super easy. Maybe we can make the webviews work, as expected, through some combination of JS injected into the client, and request interception.

We might raise issues with electron, but that's an unlikely solution as well.

[pfrazee]

Two small updates on my last post...

• I managed to get things mostly working without the standard scheme. Paths were resolving better than I expected. So, that may not be too hard to handle.
• I was pointed to Standard URI for ipfs and ipns protocols (Discussion) ipfs/kubo#1678. Reading it now.

[pfrazee]

[davidar]

I was pointed to ipfs/kubo#1678. Reading it now.

@pfraze Long story short:

• use /ipfs/Qm... wherever possible
• if you have to use a URI, use fs:/ipfs/Qm...

[davidar]

To make electron's webviews handle paths correctly (in links, in imgs, etc) we want Electron to treat the folder hash as the domain. Then, paths like /icon.png will resolve as expected

Absolute paths like /icon.png are strongly discouraged on IPFS, as they break on the HTTP gateway (and violate nesting properties). Therefore it's generally safe to assume that websites on IPFS only use relative links, with the exception of absolute /ipfs/Qm... links.

Edit: I'd actually go further than that and say that paths like /icon.png shouldn't resolve for IPFS sites, as you'd end up with webpages that work in beaker but break elsewhere.

[pfrazee]

@diasdavid hey, I'd like to do a call with the IPFS team, to hear about the vision for IPFS browser. Do you think we could schedule one?

[nicola]

hey @pfraze I am really interested. A couple of weeks ago I attempted to make an ipfs-specific browser (specific to ipfs in every sense - even its components were on IPFS and makes it easier to be self-publish), I am happy to discuss design choices, I really love the approach in combining all the decentralized projects!

great work!

[pfrazee]

@nicola right on, ping me on IRC sometime and we can chat

[daviddias]

Can we do it Tuesday, 9am PDT, 6pm UTC? @pfraze @nicola @jbenet

[pfrazee]

Are those times ==? I think they may be off by an hour.

I can do Tuesday

[nicola]

I can do tuesday too

[daviddias]

@pfraze you are right, fixed. Looking forward to chat today :)

9am PDT, 5pm WEST, 6pm UTC