Skip to content

feat: tencent vectordb: use grpc client and set upsert batch size #16016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 27, 2025
Merged

feat: tencent vectordb: use grpc client and set upsert batch size #16016

merged 1 commit into from
Mar 27, 2025

Conversation

wlleiiwang
Copy link
Contributor

@wlleiiwang wlleiiwang commented Mar 17, 2025 •
edited
Loading

Summary

We are the Tencent Vector Database team, and we hope to optimize the usage of Tencent Cloud Vector Database in Dify.

Some changes：

  • Use grpc client.
  • Set upsert batch size config
  • Update the dependency tcvectordb to the latest version.

Fix #16027

Support Tencent Vector DB
dependencies
tcvectordb==1.6.4
数据库版本>=2.0

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. dependencies Pull requests that update a dependency file 👻 feat:rag Embedding related issue, like qdrant, weaviate, milvus, vector database. labels Mar 17, 2025
@wlleiiwang wlleiiwang changed the title Tencent VectorDB: use grpc client and set upsert batch size feat: tencent vectordb: use grpc client and set upsert batch size Mar 17, 2025
@crazywoola
Copy link
Member

  • Please open an issue or link an existing issue in the description.
  • Please resolve the conflicts.

@wlleiiwang wlleiiwang force-pushed the tencent_vectordb branch 2 times, most recently from 64ce361 to 3a5aec9 Compare March 17, 2025 11:46
@wlleiiwang
Copy link
Contributor Author

  • Please open an issue or link an existing issue in the description.
  • Please resolve the conflicts.

I have resolved the conflicts.

@crazywoola
Copy link
Member

Please fix the errors in the CI.

@wlleiiwang wlleiiwang force-pushed the tencent_vectordb branch 4 times, most recently from 1130b52 to 3440d6c Compare March 19, 2025 04:07
@wlleiiwang
Copy link
Contributor Author

@crazywoola
How to pass the integration tests without using a real vector database instance address.

@crazywoola
Copy link
Member

I think you can mock the responses?

@wlleiiwang
Copy link
Contributor Author

wlleiiwang commented Mar 21, 2025 •
edited
Loading

I think you can mock the responses?

@crazywoola I found the MOCK_SWITCH and passed the test.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Mar 21, 2025
@wlleiiwang wlleiiwang force-pushed the tencent_vectordb branch 2 times, most recently from 71576b1 to 4e2e8b8 Compare March 21, 2025 06:17
@wlleiiwang
Copy link
Contributor Author

@crazywoola Please help me review again

JohnJyong
JohnJyong previously approved these changes Mar 25, 2025
View reviewed changes
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 25, 2025
@wlleiiwang wlleiiwang force-pushed the tencent_vectordb branch 3 times, most recently from c46d25e to 8beca8d Compare March 25, 2025 11:39
@wlleiiwang
Copy link
Contributor Author

rebase origin/main

@wlleiiwang
Copy link
Contributor Author

resolved conflicts

@crazywoola
Copy link
Member

Please resolve the conflicts.

@wlleiiwang
Copy link
Contributor Author

Please resolve the conflicts.

I have resolved the conflicts.

@crazywoola crazywoola merged commit a743d5d into langgenius:main Mar 27, 2025
8 checks passed
parambharat pushed a commit to parambharat/dify that referenced this pull request Mar 31, 2025
@wlleiiwang @parambharat
feat: tencent vectordb: use grpc client and set upsert batch size (la…
d9b1d14 
…nggenius#16016)

Co-authored-by: wlleiiwang <[email protected]>
jsincorporated pushed a commit to jsincorporated/asaAi that referenced this pull request Jul 8, 2025
@wlleiiwang @jsincorporated
feat: tencent vectordb: use grpc client and set upsert batch size (la…
41e5c59 
…nggenius#16016)

Co-authored-by: wlleiiwang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@crazywoola crazywoola crazywoola approved these changes

@JohnJyong JohnJyong JohnJyong left review comments

Assignees
No one assigned
Labels
dependencies Pull requests that update a dependency file 👻 feat:rag Embedding related issue, like qdrant, weaviate, milvus, vector database. lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Optimize the usage of Tencent Vector Database
3 participants
@wlleiiwang @crazywoola @JohnJyong