Gemma-4 reasoning

SambaNova Devs Discussion
1

I’m evaluating sambanova for moving my gemma-4 workload over here for fast inference.

I have two critical questions about the platform, unanswered in either docs, or playground:

  • Do you support reasoning with gemma-4? playground does not returns reasoning traces
  • EXTREMELY IMPORTANT: do you support SENDING BACK REASONING TRACES after tool call to gemma-4 AND GEMMA USING IT? this is currently borked with 90% of inference providers.

This is specifically the following case:

{
    "model": "gemma-4-31b-it",
    "messages": [
        {
            "role": "user",
            "content": "Think of a number, then send \"okay\" to me with the send_message tool, then tell me the number you thought of."
        },
        {
            "role": "assistant",
            "content": null,
            "tool_calls": [
                {
                    "id": "call_1",
                    "type": "function",
                    "function": {
                        "name": "send_message",
                        "arguments": "{\"text\":\"okay\"}"
                    }
                }
            ],
            "reasoning": "I thought of number 99. Now on to sending okay."
        },
        {
            "role": "tool",
            "tool_call_id": "call_1",
            "content": "sent"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "send_message",
                "description": "Send a chat message to the user.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "text": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "text"
                    ]
                }
            }
        }
    ],
    "temperature": 0,
    "max_tokens": 400
}

With reasoning enabled, for you as an inference provider, what does the model produce for this query.

Thank you,

-Joel

2

Hi Joel,

Thanks for reaching out and for the detailed example.

We’ve received your questions regarding Gemma 4 reasoning support and reasoning traces across tool calls. I’ll check with the team and get back to you with a more detailed response shortly.

Appreciate your patience.

3

Hi Joel,

Thank you for your questions. Happy to help with your evaluation.

Q1: Reasoning support for Gemma-4

Yes, we support reasoning with Gemma-4 via the enable_thinking parameter:

"chat_template_kwargs": {
    "enable_thinking": true
}

Here is a working curl example with tool calling:

curl --location 'https://api.sambanova.ai/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
    "model": "gemma-4-31b-it",
    "messages": [
        {"role": "user", "content": "Think of a number, then send \"okay\" to me with the send_message tool, then tell me the number you thought of."}
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "send_message",
                "description": "Send a chat message to the user.",
                "parameters": {
                    "type": "object",
                    "properties": {"text": {"type": "string"}},
                    "required": ["text"]
                }
            }
        }
    ],
    "chat_template_kwargs": {"enable_thinking": true},
    "temperature": 0,
    "max_tokens": 400
}'

The response confirms reasoning traces are returned in the reasoning field alongside the answer:

{
  "message": {
    "role": "assistant",
    "reasoning": "The user asked me to think of a number, send 'okay' using the send_message tool, and then tell them the number. I have already sent 'okay'. Now I need to reveal the number I thought of. I thought of the number 42.",
    "content": "I thought of the number 42."
  }
}

Q2: Reasoning trace persistence across tool calls

We have raised internal ticket and investigating this and will follow up with you shortly.

Best regards,
Virendra Jopale