500 internal server error

SambaNova Devs
1

Hello
I am a new with SambaNova Fast Api.
I tried your draft code to check my test api key but getting the 500 internal server error and couldn’t get the result
I will attach my code.

SAMBANOVA_API_KEY = “”

from openai import OpenAI
client = OpenAI(
base_url=“https://fast-api.snova.ai/v1/chat/completions”,
api_key=SAMBANOVA_API_KEY,
)
model = “llama3-405b”
prompt = “Tell me a joke about artificial intelligence.”

completion = client.chat.completions.create(
model=model,
messages=[
{
“role”: “user”,
“content”: prompt,
}
],
stream=True,
)

response = “”
for chunk in completion:
response += chunk.choices[0].delta.content or “”

print(response)

2 Likes
2

can you try it with the base URL set to: “https://fast-api.snova.ai/v1”

1 Like
3

I verified your code works with Vasanth’s modification

from openai import OpenAI
client = OpenAI(
 base_url='https://fast-api.snova.ai/v1',
api_key='obfuscated',
)
model = 'llama3-405b'
prompt = 'Tell me a joke about artificial intelligence.'

completion = client.chat.completions.create(
model=model,
messages=[
    {
    'role': 'user',
    'content': prompt,
     }
],
stream=True,
)
response = ''
for chunk in completion:
  response += chunk.choices[0].delta.content or ''

print(response)

>>> print(response)
Here's one:

Why did the AI program go to therapy?

Because it was struggling to process its emotions!

I hope that one computed to a laugh!
2 Likes
4

Yes! it works for me.
Thank you @cody and @Vasanth!
By the way, can I use this test api key in production mode?
Could you share me the price plan?

1 Like
5

Hi,
Could you please reach out to me at sarosh.naseem@sambanovaystems.com. Happy to discuss the pricing tiers with you.

Regards,
Sarosh Naseem
510.566.9229

6

Hi
The email address you sent is not wokring.
Can you send me another way to discuss please?
BR,
Heorhii

1 Like
7

Herohii,

I apologize there was a typo in the above email address . Please use : sarosh.naseem@sambanovasystems.com . The one above was missing the first s in systems .

Regards

Coby

1 Like
8

Hi, @cody and @Vasanth!
Could you let me know how many tokens I can use in test mode with the given api key?
Or the test api key expires in certain time?
I got the expiration alert once yesterday, and after that getting lower speed without any alert.
Please let me clear about this.
Thanks

9

@suslovwebhero I am so sorry for the delay . With the free tier the token count rate is per minute so it’s advisable to set your code up to handle the 429. The speed variance can be contributed to demand on the free tier assets . The developer tier will be coming soon and the enterprise tier is already available to purchase .

As to the specific rate limit items for the free tier we will be publishing those numbers soon.

-Coby

10

Hi. Thanks for your replies.

We send several emails to sarosh.naseem@sambanovasystems.com about the pricing tiers but without success.
Woulds it be possible to move forward on this? We’re eager to move to a paying/custom tier so we can validate the real time user case which will make a significant difference for our project. Going real time would in fact be a game changer for our scaling.

Best regards,
JX

11

@jxbersot I apologize for any disconnect. I have reached out to see if the messages have routed .

-Coby

1 Like
12

@coby.adams thanks!

Quick additional question: SambaNova is using OpenAI API to function. Does that imply any transfer of data to OpenAI? Any copyright or IP going to OpenAI in the process?

1 Like
13

@jxbersot No we do not send any data to OpenAI. The use of their package is simply to make the moving of applications from other API vendors to ours easier. If you look at most AI endpoint vendors they use the OpenAI package.

1 Like
14

@coby.adams Thanks for your replies.

Any update on the pricing part? We need it to move to higher limits and go on production. We’re on stand by because of this for a while now. Thanks for your help!

15

@jxbersot I sent you a DM and looped in sales.

-Coby