Utility - Print your rate limits

SambaNova Devs Discussion
1

Hey All,

Just a quick utility post! Print out your assigned limits per model.

Setup

  1. Install dependencies:

pip install httpx prettytable

  1. Set your API key:
export SAMBANOVA_API_KEY='your_api_key_here'

Usage

Run the script:

import os
import httpx

# Load API credentials
api_key = os.environ.get("SAMBANOVA_API_KEY")
if not api_key:
    raise ValueError("SAMBANOVA_API_KEY environment variable not set")

base_url = "https://api.sambanova.ai/v1"
headers = {"Authorization": f"Bearer {api_key}"}


def get_available_models():
    """Fetch available models from the API"""
    response = httpx.get(f"{base_url}/models", headers=headers)
    response.raise_for_status()
    return [model["id"] for model in response.json()["data"]]


def get_request_limit(model_id):
    """Get request rate limit for a specific model"""
    data = {
        "model": model_id,
        "messages": [{"role": "user", "content": "."}],
        "temperature": 0.1,
        "top_p": 0.1,
    }
    response = httpx.post(f"{base_url}/chat/completions", headers=headers, json=data)
    return response.headers.get("x-ratelimit-limit-requests", "N/A")


def main():
    models = get_available_models()
    print("Model Request Limits:")
    print("---------------------")
    for model in models:
        try:
            limit = get_request_limit(model)
            print(f"{model}: {limit}")
        except Exception as e:
            print(f"{model}: Error - {str(e)}")


if __name__ == "__main__":
    main()

Example Output

Model Request Limits:
---------------------
DeepSeek-R1-0528: 60
DeepSeek-R1-Distill-Llama-70B: 240
DeepSeek-V3-0324: 60
...

Notes

  • Requires valid Sambanova API credentials
  • Makes API calls to:
    • GET /v1/models to get available models
    • POST /v1/chat/completions to get rate limit headers
1 Like