How to return token usage in self-develop plugin

[shua-chen]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

My dify version is 1.9.2. I am developing a plugin and it requires user to select one llm model to user it. But in my test, user cannot see the token usage caused by llm, I wonder how I can add this or it is a bug in current version.

2. Additional context or comments

No response

[dosubot[bot]]

To return token usage in your self-developed plugin for Dify v1.9.2, your plugin must ensure that the LLM call returns usage data in the expected format and that Dify is configured to collect it.

1. For plugin-based or custom models, set the environment variable PLUGIN_BASED_TOKEN_COUNTING_ENABLED=true to enable token usage counting. This is required for Dify to track tokens for plugin-based models, but not for standard LLM nodes like GPT-4 or Claude [source].

2. For streaming LLM calls, you must include stream_options={"include_usage": True} in your model call parameters. Without this, token usage will always be zero, even if the LLM returns output. In your plugin’s _invoke method, add:

   if stream:
       model_parameters["stream_options"] = {"include_usage": True}

   before making the LLM call or calling the superclass. This ensures the model’s native usage reporting is used [source].

3. For custom LLM models, implement the get_num_tokens method to return the token count for the given prompt. If your model can’t provide this directly, you can use the GPT2 tokenizer-based fallback if PLUGIN_BASED_TOKEN_COUNTING_ENABLED is set [source].

4. Token usage is only updated if nonzero: Dify will only show token usage in the UI/logs if your plugin/model returns usage data with total_tokens > 0 [source].

Troubleshooting:
If you still see zero tokens after these changes, check that:

• Your plugin/model actually returns usage data in the expected format.
• The workflow reaches the LLM invocation step.
• The configuration flag is set and your plugin integration supports usage reporting [source].

This is a known limitation and not a bug in Dify itself, but rather a requirement for plugin implementation. No recent PRs have changed this behavior, so manual code changes are currently the best solution [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}