use tool is slow

[wanshichenguang]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.3.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Using custom tools in chatflow or agent is very slow. When I call them with Postman or Python code, I get the result about 100ms, but in Dify it takes 5 seconds to get the result.

✔️ Expected Behavior

Speed up the tool usage.

❌ Actual Behavior

Using custom tools in chatflow or agent is very slow. When I call them with Postman or Python code, I get the result about 100ms, but in Dify it takes 5 seconds to get the result.

[wanshichenguang]

[dosubot]

I found some similar discussions regarding slow execution of tools in Dify:

1. Using post to call dify chatflow's API (enable streaming), the response is very slow - This discussion is still open and might provide insights into similar issues.
2. When I use Dify to orchestrate an agent to invoke tools, the API response times out because my tools are too slow - This is another open discussion that might be relevant.
3. The HTTP request node responds slowly - This discussion is also open and could be related to your issue.

For optimizing performance in a self-hosted Dify setup using Docker, consider the following configurations:

• Server and Worker Configuration: Adjust the SERVER_WORKER_AMOUNT to match your CPU cores using the formula CPU cores x 2 + 1 for optimal performance. Also, consider increasing the GUNICORN_TIMEOUT to support long SSE connections ^[1].

• Concurrency Management: Dify uses a GraphEngineThreadPool to manage concurrent executions. The default MAX_SUBMIT_COUNT is set to 100, which allows for a higher number of concurrent task submissions. Adjusting these settings might help improve performance ^[2].

• Timeout Settings: Check the timeout settings in your config/provider.json file. The default is 120 seconds, but you might need to adjust this based on your specific use case ^[3].

These configurations and settings can help optimize the performance of custom tools in Dify. Adjust them based on your specific hardware and workload requirements.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}