Run demo program in Open AI Compatibility
from openai import OpenAI
client = OpenAI(
base_url=“https://api.sambanova.ai/v1”,
api_key=“YOUR SAMBANOVA API KEY”
)response = client.chat.completions.create(
model=“Meta-Llama-3.1-8B-Instruct”,
messages=[
{“role”: “system”, “content”: “Answer the question in a couple sentences.”},
{“role”: “user”, “content”: “Share a happy story with me”}
]
)print(response.choices[0].message)
The output is
ChatCompletionMessage(content=“A 9-year-old boy from Philadelphia started a lemonade stand to raise money for his neighbor who was struggling to pay her medical bills. He ended up raising over $50,000 and his act of kindness inspired many others in the community to help, ultimately changing the neighbor’s life for the better.”, refusal=None, role=None, function_call=None, tool_calls=None)
The ‘role’ property is None. This will broke langchain-openai, and inconsistent with SambaNova Cloud API Reference which the sample output contains ‘role’ property