Init check fails but version does exist (migrating 9 -> 10)

[v-stickykeys]

Hello!

I'm trying to migrate a repo from v9 to v10. It uses s3 as a backend for ipfs-repo.

Before migration attempt

The repo clearly has version and config files.

λ cat version 
9

During migration attempt

The migration is failing and shows the following logs:

2021-04-08T11:36:27.731Z ipfs:repo opening at: ipfs
--
2021-04-08T11:36:28.234Z ipfs:repo init check
2021-04-08T11:36:28.459Z ipfs:repo acquired repo.lock
2021-04-08T11:36:28.713Z ipfs:repo:version comparing version: 9 and 10
2021-04-08T11:36:29.233Z ipfs:repo migrating to version 10

2021-04-08T11:36:29.787Z ipfs:repo:migrator:repo:init Version entry present: false # <---- WHY??
2021-04-08T11:36:29.787Z ipfs:repo:migrator:repo:init Config entry present: false # <---- WHY??

NotInitializedRepoError: Repo in path ipfs is not initialized!
at Object.getVersion (/ipfs-repo-migrations/src/repo/version.js:22:11)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async Object.migrate (/ipfs-repo-migrations/src/index.js:67:26)
at async IpfsRepo.open (/ipfs-repo/src/index.js:120:11)
at async openRepo (/ipfs-core/src/components/storage.js:88:7)
at async loadRepo (/ipfs-core/src/components/storage.js:60:21)
at async Function.start (/ipfs-core/src/components/storage.js:44:41)
at async Object.create (/ipfs-core/src/components/index.js:208:21)
at async Function.create (/ipfs-daemon.js:65:22) {
code: 'ERR_NOT_INITIALIZED_REPO'
}

After migration attempt

The repo gets 2 new files which seem like they should not be there and version is still on 9

---

Could the issue be here?

js-ipfs-repo/src/version.js

Lines 26 to 31 in a36e695

	async exists () { // eslint-disable-line require-await
	// level-js@5.x cannot read keys from level-js@4.x dbs so fall back to
	// using IndexedDB API with string keys - only necessary until we do
	// the migratiion to v10 or above
	return hasWithFallback(versionKey, store.has.bind(store), store)
	},

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[v-stickykeys]

For context we have updated our dependencies from

// prev versions
"ipfs": "0.52.2",
"ipfs-http-gateway": "^0.3.0",
"ipfs-http-server": "0.3.1",

to

// curr versions
"ipfs": "~0.54.2",
"ipfs-http-gateway": "~0.3.2",
"ipfs-http-server": "~0.3.3"

[achingbrain]

Could you check the config and see if the root datastore is being sharded? I think what may be happening here is when a repo is first initted the basic structure is created - the blocks dir, version file, etc - without any sharding. Sharding is applied per-datastore so doesn't need to be applied to the root.

When the repo-migrations tool tries to read the version number it applies whatever datastore options have been passed - if you're using the default createRepo script bundled with datastore-s3 it'll try to shard everything which is a bug - it'll try to load the version/config files with a shard prefix which will obviously fail as the root wasn't created in a sharded fashion.

I have updated the file in ipfs/js-datastore-s3#33 to only shard the blockstore and the pinstore as the other datastores should be small enough that sharding isn't necessary, and the root itself should never be sharded.

[v-stickykeys]

Hey @achingbrain thanks for looking into this. We are creating the repo like so

const repo = configuration.ipfsS3RepoEnabled ? createRepo({
            path: configuration.ipfsPath,
        }, {
            bucket: configuration.awsBucketName,
            accessKeyId: configuration.awsAccessKeyId,
            secretAccessKey: configuration.awsSecretAccessKey,
        }) : configuration.ipfsPath

Current config (pre-migration of course) is this

"datastore": {
    "Spec": {
      "type": "mount",
      "mounts": [
        {
          "mountpoint": "/blocks",
          "type": "measure",
          "prefix": "flatfs.datastore",
          "child": {
            "type": "flatfs",
            "path": "blocks",
            "sync": true,
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2"
          }
        },
        {
          "mountpoint": "/",
          "type": "measure",
          "prefix": "leveldb.datastore",
          "child": {
            "type": "levelds",
            "path": "datastore",
            "compression": "none"
          }
        }
      ]
    }
  },

So my guess is that this bug is affecting us.

[achingbrain]

Have you managed to perform the migration? It should be a case of using the (updated) create-s3-repo.js file from datastore-s3 - it's been moved to the examples folder so you'll need to copy it into your project and ensure you have a version of ipfs-repo in your deps that matches the version ipfs is using.

The only thing to double check would be which datastores are sharded - if you look in the blocks, datastore, keys and pins folders of your s3 bucket they'll be sharded if they are full of very short-named folders, otherwise not.

[v-stickykeys]

@achingbrain thanks for merging this fix. I'll be testing it within the next couple days as we copy over the create repo code and let you know how it goes.

[lidel]

@valmack feel free to reopen if still an issue in the latest release.