Migrate object storage clients to use thanos objstore.Bucket implementations

[jtlisi]

Currently we have generic ObjectClient interface defined here:

cortex/pkg/chunk/storage_client.go

Lines 64 to 71 in 58c081a

	// ObjectClient is used to store arbitrary data in Object Store (S3/GCS/Azure/Etc)
	type ObjectClient interface {
	PutObject(ctx context.Context, objectKey string, object io.ReadSeeker) error
	GetObject(ctx context.Context, objectKey string) (io.ReadCloser, error)
	List(ctx context.Context, prefix string) ([]StorageObject, error)
	DeleteObject(ctx context.Context, objectKey string) error
	Stop()
	}

This interface is used by the pkg/chunk/objectclient to store chunks. It will also be used by @sandeepsukhani works on delete. I also utilize this interface for my rules/alerts branches. These clients are pretty generic and I think we might be able to utilize some off the shelf libraries instead of rolling our own object client. I found two potential libraries that might work:

1. Go Cloud Development Kit

https://github.com/google/go-cloud

Keeps things very generic and easy for reuse.

2. Reuse Thanos
   The Thanos libary is pretty full featured and would be able to do most of what we want. It would also be nice for unifying the two ecosystems.
   https://github.com/thanos-io/thanos/blob/master/pkg/objstore/objstore.go

[pracucci]

Reuse Thanos

Unless we have a good argument to not use the Thanos objstore I would use it. We're already using it for the blocks storage anyway, so it's already vendored.

[jtlisi]

@pracucci I agree. I added some tags to signal to anyone who wants to take up this issue.

[jaybatra26]

@jtlisi and @pracucci Can I take this up as my next work item?

[jtlisi]

I think it would be fine if you take this up. Please use the thanos object store if you can.

You will need to make sure data created with the current clients is accessible with the new client so be sure to use good test cases.

Also let me touch base with the people working on Loki since they use these libraries as well.

[jaybatra26]

Thanks @jtlisi . I am assuming that I would have to write something like this code snippet for each of the object store i.e(s3/gcs/azure etc).
Also, looks like local file system is in testing mode.

Is that correct?

[jtlisi]

@jaybatra26 In this case you should be able to reuse that snippet and update the pkg/chunk/objectclient code to utilize the objstore.Bucket interface. You will also have to update anything that utilizes the chunk.ObjectClient interface to use the objstore.Bucket interface instead. This includes the code in pkg/chunk/purger and probably the pkg/ruler/rules/objectclient implementation if #2269 is merged.

It may make sense to move the packages in pkg/storage/tsdb/backend to the more general storage/backend path. @pracucci does that sound good to you?

[pracucci]

It may make sense to move the packages in pkg/storage/tsdb/backend to the more general storage/backend path. @pracucci does that sound good to you?

It makes sense to me. However, I would advocate for incremental PRs. A first PR may be which package move, then we may have a PR for the object store client interface refactoring and then a PR for each object store client replacement?

[jaybatra26]

Thanks @jtlisi and @pracucci. As @pracucci suggested, I will create incremental PR's.