Unable to load plugin

[ianopolous]

I've built this plugin and used it with ipfs v0.4.23 and it works fine. However, When I rebuild ipfs v0.4.23 with the 2 line diff from ipfs/go-ipfs-pinner#2 I get the following error:

Error: error loading plugins: loading plugin /tmp/peergos9005038333179153292/.ipfs/plugins/go-ds-s3.so: plugin.Open("/tmp/peergos9005038333179153292/.ipfs/plugins/go-ds-s3"): plugin was built with a different version of package github.com/jbenet/goprocess

@Stebalien Any idea what could be happening here? That goprocess repo hasn't changed in almost a year.

[ianopolous]

I also haven't been able to get this to build against master of go-ipfs:

make IPFS_VERSION=/home/ian/go/src/github.com/ipfs/go-ipfs

./set-target.sh /home/ian/go/src/github.com/ipfs/go-ipfs
go: cannot find main module, but found Gopkg.lock in /home/ian/go-ds-s3
to create a module there, run:
go mod init
Makefile:20: recipe for target 'go.mod' failed
make: *** [go.mod] Error 1

[Stebalien]

@Stebalien Any idea what could be happening here? That goprocess repo hasn't changed in almost a year.

I'm not sure but go is weird about plugins. Setting IPFS_VERSION=/path/to/build should fix it. Alternatively, make sure nothing has changed in any go.mod files.

Alternatively, you can build your plugin into go-ipfs by following the directions here https://github.com/ipfs/go-ipfs/blob/master/docs/plugins.md#preloaded-plugins.

I also haven't been able to get this to build against master of go-ipfs:

Assuming your current working directory is go-ds-s3, did you delete the go.mod file?

[ianopolous]

@Stebalien Thanks for your help. I've tried again starting from scratch in both repos.

First, in go-ipfs I checked out release/v0.4.23 and applied the 2 line patch for the global pin lock
That builds fine.

Then in go-ds-s3 I checked out release/v0.4.23 and ran

make IPFS_VERSION=/home/ian/go/src/github.com/ipfs/go-ipfs

That builds fine now (I think you're right - I must have deleted go.mod), but then when I try to load it in the ipfs built in step 1 I get the following from ipfs init:

Error: error loading plugins: loading plugin /tmp/peergos-s3-test/.ipfs/plugins/go-ds-s3.so: plugin.Open("/tmp/peergos-s3-test/.ipfs/plugins/go-ds-s3"): plugin was built with a different version of package github.com/jbenet/goprocess

This is a huge problem for us because we can't upgrade ipfs, so to avoid a global deadlock we have to disable gc entirely.

[ianopolous]

Is it possible to strip down the amazon s3 lib such that it could be included in ipfs by default? I'm not sure if the go compiler does that, but based on the size of the .so I'd guess not. I was able to reduce the Java amazon S3 library down to 2.8mb, which would be 6% of the current ipfs binary size.

[ianopolous]

This looks very similar to what I'm seeing. I'm going to try some of their ideas: golang/go#27751

UPDATE: none of that works.

[ianopolous]

I've even gone so far as to try forking go-ipfs, renaming the module, adding the patch, pushing a new version tag to my repo, and then trying linking the s3 plugin to that. (in case the error was some issue with using a local go-ipfs version rather than a remote)

Sadly that results are the same goprocess error when ipfs tries to load the module.

[Stebalien]

Yeah, I'm seeing that error. GAH!

[Stebalien]

TL;DR: Checkout the feat/peergos-0.4.23-patched in go-ipfs and build that. That will compile the s3 datastore into go-ipfs at the same time.

---

Ok, I found the issue.

• In dist.ipfs.io, we strip the gopath from paths on build.
• We do the same in go-ds-s3.
• We do the same in go-ipfs master.
• We don't do this in go-ipfs v0.4.23 (never backported the patch).

GOFLAGS += -asmflags=all=-trimpath="$(GOPATH)" -gcflags=all=-trimpath="$(GOPATH)"

Adding this to the 0.4.23 makefile fixes the problem. But really, just use the branch.

[ianopolous]

Ah, thank you so much! You're a lifesaver! That fixes it. I've patched the go-ipfs makefile with that and everything works now.

We've built our ipfs installer around optionally loading plugins, so didn't want to go down the "all in one" route.

Thanks again!