Overhaul datastore with query fixes, logging, and tests #304
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While I was trying to find a reason why s3 store started losing blocks (ipld block not found error). I've played around with the repo and it seems that after few guided vibe-coding iterations it finally became stable. There were several issues with pagination/key handling logic which were addressed.
Potentially this fixes #261. At least it fixed it for me on my node
Key changes:
Correctness Fixes:
Query
method has been completely rewritten to be a stateful iterator.ListObjectsV2
pages. The previous implementation was limited to a single page of results (max 1000 objects).Offset
andLimit
are now properly supported across paginated results.Delete
operations now correctly prefix keys with the configuredRootDirectory
before sending them to S3.go-datastore
conventions.Refactoring & Testability:
s3iface.S3API
interface.Put
andDelete
)Offset
, andLimit
Observability & Dependencies:
go-log/v2
to all key datastore operations (Put
,Get
,Query
,Batch
, etc.).kubo
andgo-datastore
, and bumped the Go version to1.25
..gitignore
with common editor and OS patterns.