Testing API KEY Failed

SambaNova Devs Discussion
1

Hello, I recently wanted to try a tool that requires an API KEY call for models, so I opted to try the SambaNova meta llama 3.3 70B instruct and upon testing, I got this error:

BadRequestError: litellm.BadRequestError: SambanovaException - Unknown model: meta-llama-3.1-70b-instruct

the tool is openhands, an agent-based AI, does anyone have any idea about this?

if it is related to limits is there a way to contact sambaNova support for a limit increase for a testing window of 2 days?

2 Likes
2

hi @yosri.bensaad, can you try giving model name as “Meta-Llama-3.3-70B-Instruct”

Thanks

1 Like
3

hi @omkar.gangan, i made a mistake in my post it is the 3.1 not 3.3 (i’ll edit it later) but the model is already named with - format, here’s a screenshot.

image
image402×264 8.69 KB

2 Likes
4

Can you please try these

image
image966×1354 76.4 KB

Its working for me

image
image998×1364 86.7 KB

Thanks & Regards

2 Likes
5

@yosri.bensaad As Omkar has stated you should be able to specify the value in that box .

here is an examople method of doing ti from scratch on a MAC .

Sambanova Cloud Local Agents (Openhands)
Create your own code agents that can interact with the world and use Sambanova Cloud for a blazing fast experience with large models.

Macbook Setup (LIMA)

  1. Install LIMA (Linux VMs on Mac)
    brew install lima
  2. Create A Virtual Machine (Virtual machine with VZ virtualization and rosetta to translate AMD64 to ARM64 instructions)
    limactl start --cpus=2 --memory=6 --vm-type=vz --rosetta --name=ubuntu-amd64
  3. Connect to The Virtual Machine
    limactl shell ubuntu-amd64
  4. Install Docker
    sudo apt update && sudo apt install -y docker.io
    Run OpenHands Locally
  5. Connect to VM if using LIMA (limactl shell ubuntu-amd64)
  6. Pull the runtime container
    sudo docker pull docker.all-hands.dev/all-hands-ai/runtime:0.18-nikolaik
  7. Run the OpenHands Agent Container
  8. sudo docker run \
  9.  -it \

  10.  --rm \

  11.  --pull=always    \

  12.  -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.18-nikolaik \

  13.  -e LOG_ALL_EVENTS=true \

  14. -v /var/run/docker.sock:/var/run/docker.sock \

  15. -v ~/.openhands-state:/.openhands-state \

  16. -p 3000:3000 \

  17. --add-host host.docker.internal:host-gateway \

  18. --name openhands-app \
    docker.all-hands.dev/all-hands-ai/openhands:0.18
  19. Open your web browser to http://localhost:3000
  20. Setup the API Provider
  21. Click on the Settings Icon
  22. Enter a model (Use sambanova/Meta-Llama-3.1-405B-Instruct for a good starting point)
  23. Enter your API key from SambaNova Cloud
  24. Click “Save”

image
image1264×1218 105 KB

  1. Enter your prompt and start building

If the VM was set up for you then they need to modiffy the environment variables they used for the LLMs to ensure proper model name formatting for SambaNova

-Coby

2 Likes
6

thank you both for the reply, i’ll try them and update you if there was any change.

2 Likes