Function Calling & JSON Mode in SambaNova Cloud

Knowledge Articles
,
1

Overview:

SambaNova Cloud supports advanced function-calling capabilities that enable structured, tool-augmented responses. This allows you to build more dynamic, context-aware applications that can interface directly with external APIs and tools.

:wrench: Function Calling — How It Works:

Function calling allows a model to decide whether to answer a prompt directly or suggest calling a tool/function defined by the user. Here’s the typical flow:

  1. Define Tools: Describe functions using JSON Schema (name, parameters, types, and descriptions).
  2. Submit Query: Send user prompt + tool definitions.
  3. Model Determines Action:
  • Responds directly OR
  • Fills in a function name and arguments.
  1. User Executes the Function (e.g., API call, DB query).
  2. Pass Result to Model: Feed the output back for formatting or follow-up questions.

:white_check_mark: Supported Models:

Model Name Notes
meta-llama-3.1-8b-instruct Suitable for one-shot tool use.
meta-llama-3.1-405b-instruct Fully supports function-calling.
meta-llama-3.3-70b-instruct Optimized for tool-augmented interactions.
llama-4-scout-17b-16e-instruct Supports both JSON and function mode.
deepseek-v3-0324 Also function-call aware and JSON-capable.

:test_tube: Example Python Workflow

from openai import OpenAI

client = OpenAI(base_url="https://api.sambanova.ai/v1")

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get weather for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {"type": "string", "description": "City name"},
        },
        "required": ["location"]
      }
    }
  }
]

response = client.chat.completions.create(
    model="meta-llama-3.3-70b-instruct",
    messages=[
        {"role": "user", "content": "What's the weather in Mumbai?"}
    ],
    tools=tools,
    tool_choice="auto"  # or "required" or {"type": "function", "function": {"name": "get_weather"}}
)

# You then execute the suggested tool call, capture its result, and send it back in follow-up messages.

:page_facing_up: JSON Output Modes

1. json_schema (Structured)

  • Forces output that matches a specific JSON schema.
  • Use response_format={"type": "json_schema", ...}.
  • Useful for information extraction, formatting, or downstream API calls.

2. json_object (Any Valid JSON)

  • Guarantees valid JSON object output.
  • Use response_format={"type": "json_object"}.
  • Helpful when structure matters but schema isn’t strictly enforced.

:blue_book: Use Cases

  • Weather apps
  • Financial data lookups
  • Product detail extraction
  • Multi-step agent frameworks
  • Dynamic prompt-to-API pipelines
4 Likes