DeepSeek-V3.1 - An approach for thinking and tool use

SambaNova Devs Discussion
1

Hey Samba Speed Freaks,

3.1 is hot off the press and here’s an approach to getting some tool use and thinking at the same time from it. Although the model doesn’t explicitly support it, I’ve created this method while we work on improving our api and templates to make it seamless for everyone.

For enabling thinking one can use this method in the chat client init:
extra_body={"chat_template_kwargs": {"enable_thinking": True}},

However, I took a stab at modifying the system prompt to accomplish the same and use tools at the same time. This script is interactive and allows the user to specify /debug to see the full agent response printed in the console, and /think to enable or disable thinking.

Let me know if you like this approach or how you are accomplishing these tasks in your own ways.

Thanks
Seth

tools.py (6.4 KB)

deepseek-v3.1-tools-thinking-toggle.py (7.1 KB)

3 Likes
2

Great this was a big help - really helped to implement with tool calling

What we saw was DeepSeek V3.1 (accessed through SambaNova) uses a native XML format for tool calls instead of the OpenAI JSON standard. The model outputs tool invocations like this:

<tool_call>
{"tool_name": "web_search", "tool_arguments": {"query": "latest AI news"}}
</tool_call>

so adding XML detection and parsing was the way we got around it - and now its working :slight_smile:

but we did this wihtout the thinking tokens ….will look at that later

3 Likes