Hi
I signed up for Fast API and received “Personal API Key” and " API Endpoint". I plugged those two into OpenAI Python Quick Start Guide (OpenAI Python Quick Start Guide). However, when I run the program (in CoLabs for context), I get the following error:
AuthenticationError:
401 Authorization Required
nginx
And, here is the error exception stack:
AuthenticationError Traceback (most recent call last)
in <cell line: 24>()
22 sys_prompt = “Answer the question in a couple sentences.”
23 prompt = “Share a happy story with me”
—> 24 out = call_llama(sys_prompt, prompt)
25 print(out)
5 frames
/usr/local/lib/python3.10/dist-packages/openai/_base_client.py in _request(self, cast_to, options, remaining_retries, stream, stream_cls)
1039
1040 log.debug(“Re-raising status error”)
→ 1041 raise self._make_status_error_from_response(err.response) from None
1042
1043 return self._process_response(
For Reference, here is the code – Note: no changes other than updating API key and URL. I intentionally corrupted the API key for this post
import openai
from openai import OpenAI
SAMBANOVA_API_URL = “https://fast-api.snova.ai/v1/chat/completions”
SAMBANOVA_API_KEY = “skhsakhfkYaldQyajaWwuY29tAKHKasssZ1B2dlI=” # corrupted key
client = OpenAI(
base_url=SAMBANOVA_API_URL, # the endpoint IP running on vLLM
api_key=SAMBANOVA_API_KEY,
)
def call_llama(sys_prompt, prompt, **kwargs):
completion = client.chat.completions.create(
model=“Meta-Llama-3.1-8B-Instruct”,
messages=[
{“role”: “user”, “content”: prompt}
],
stream=True,
**kwargs,
)
response = “”
for chunk in completion:
response += chunk.choices[0].delta.content or “”
return response
sys_prompt = “Answer the question in a couple sentences.”
prompt = “Share a happy story with me”
out = call_llama(sys_prompt, prompt)
print(out)