Getting "model does not exist or don't have access to model" errors for all models?

SambaNova Devs Discussion
1

I am trying to do text completion calls via the API. I’ve tried a few of the model IDs and I always get an error message like:

Error Unknown status code: 404 The model `Meta-Llama-3.1-8B-Instruct` does not exist or you do not have access to it

Can someone tell me how to fix this? I am able to use those models from the playground, but not via an OpenAI call, which I thought was code compatible.

Note, I definitely can make calls if I do a straight axios post call, so I although SambaNova is OpenAI compatible at the base level, I suspect that there’s a problem with the OpenAI NPM package. This code works fine in my Node.js server:

async function testSambaNova(baseUrl: string, apiKey: string) {
	try {
		const response = await axios.post(
			baseUrl,
			{
				messages: [
					{ role: "system", content: "Answer the question in a couple sentences." },
					{ role: "user", content: "Share a happy story with me" }
				],
				stop: ["<|eot_id|>"],
				model: "Meta-Llama-3.1-8B-Instruct",
				stream: false,
				stream_options: { include_usage: true }
			},
			{
				headers: {
					"Authorization": `Bearer ${apiKey}`,
					"Content-Type": "application/json"
				}
			}
		);

		return response.data;
	} catch (error) {
		console.error("Error:", error);
		throw error;
	}
}

And I also can make calls via the CURL example shown here:

But not from the API if I use the OpenAI npm PACKAGE, with the SambaNova URL as the baseURL parameter (see code below).

I turned Node HTTP debugging on and everything seems to be fine in the actual outbound call:

ath: null,
  href: 'https://api.sambanova.ai/v1/chat/completions',
  method: 'POST',
  headers: [Object: null prototype] {
    'content-length': [ '2605' ],
    accept: [ 'application/json' ],
    'content-type': [ 'application/json' ],
    'user-agent': [ 'OpenAI/JS 4.63.0' ],
    'x-stainless-lang': [ 'js' ],
    'x-stainless-package-version': [ '4.63.0' ],
    'x-stainless-os': [ 'Windows' ],
    'x-stainless-arch': [ 'x64' ],
    'x-stainless-runtime': [ 'node' ],
    'x-stainless-runtime-version': [ 'v22.0.0' ],
    authorization: [ 'Bearer <REDACTED' ],
    'x-stainless-retry-count': [ '0' ],
    'Accept-Encoding': [ 'gzip,deflate' ]
  },

But no luck.

Here is my code for the call:

export const oai = new OpenAI({
	baseURL: "https://api.sambanova.ai/v1",
	apiKey: process.env.SAMBANOVA_API_KEY,
})

		const response = await oai.chat.completions.create({
			model: textCompletionParams.model_param_val,
			messages,
			// NOTE: Currently this parameter is ignored.  Use the repetition
			//  penalty instead.
			frequency_penalty: textCompletionParams.frequency_penalty_param_val,
			// NOTE: Currently this parameter is ignored.  Use the repetition
			//  penalty instead.
			presence_penalty: textCompletionParams.presence_penalty_param_val,
			stream: false, // Set stream to false for immediate response
			temperature: textCompletionParams.temperature_param_val,
			max_tokens: textCompletionParams.max_tokens_param_val,
			top_p: textCompletionParams.top_p_param_val,
		});

Here is the contents of the textCompletionParams object.

image
image1127×432 41 KB

2

I use https://fast-api.snova.ai/v1 end point with the OpenAI client and all models work for me, can you try this instead?

1 Like
3

I thought this was the URL for SambaNova completions?:

https://api.sambanova.ai/v1

4

Just tried https://api.sambanova.ai/v1 endpoint with OpenAI package and works fine for me. Check your envs once to see if it’s overriding the base end point OPENAI_API_KEY