SambaNova Cloud Quick Start Guide

SambaNova Documentation SambaNova Cloud
1

Get started using the SambaNova Cloud API in just a few minutes!

Step 1: Get your API key

Visit the API section of the SambaNova Cloud portal to generate your API key.

Step 2: Pick a model

The SambaNova cloud offers the latest open source models running at the fastest tokens per second.

To make an inference request from a model, you’ll need the model’s corresponding model-id. The full list of models and corresponding model-id’s are listed here

We’ll use Meta’s Llama 3.1 8B as an example for the remainder of this guide.

Step 3: Make an API request

There are two ways you can make an inference request:

  1. Using a CURL command.
  2. Via Python using the OpenAI client library.

Learn more about the SambaNova Cloud API here

CURL command

In a terminal, run the CURL command below to make your first request to the API.

export API_KEY=<YOUR API KEY>
export URL=https://api.sambanova.ai/v1/chat/completions

curl -H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "system", "content": "Answer the question in a couple sentences."},
{"role": "user", "content": "Share a happy story with me"}
],
"stop": ["<|eot_id|>"],
"model": "Meta-Llama-3.1-405B-Instruct",
"stream": true, "stream_options": {"include_usage": true}
}' \
-X POST $URL

OpenAI client library

  1. To get started, ensure you have Python 3 and pip installed. Then, install the OpenAI library using pip, as shown below.
pip install openai
  1. Next, copy the code below into a python file called, hello_world.py.
from openai import OpenAI

client = OpenAI(
    base_url="https://api.sambanova.ai/v1", 
    api_key="YOUR SAMBANOVA API KEY"
)

completion = client.chat.completions.create(
  model="Meta-Llama-3.1-405B-Instruct",
  messages = [
      {"role": "system", "content": "Answer the question in a couple sentences."},
      {"role": "user", "content": "Share a happy story with me"}
    ],
  stream= True
)

for chunk in completion:
  print(chunk.choices[0].delta)
  1. Once copied into the hello_world.py file, replace the string field in SAMBANOVA_API_KEY with your API Key. Then run the file as shown below.
python hello_world.py
  1. After you run the program, you should now see an output like similar to the one below.

Here’s a happy story: One day, a little girl named Sophie found a lost puppy in her neighborhood and decided to take it home to care for it. As she nursed the puppy back to health, she named it Max and the two became inseparable best friends, going on adventures and playing together every day.

Next Steps

SambaNova AI Starter Kits are a collection of open-source examples and guides to facilitate the deployment of AI-driven use cases in the enterprise. The Enterprise Knowledge Retriever Kit allows you to quickly build a semantic search RAG application with Langchain and Streamlit in Python.

To build a complete Enterprise knowledge retriever app, click on this AI Starter Kit

3 Likes