Got access to deepseek r1 but unable to use it on cline and openwebui

[abkjay]

Thank you for the early access to the DeepSeek-R1 671B model. I’m excited to start using it.

However, I’m encountering an issue when trying to use the API endpoint within the VS Code extension “cline.” I’m receiving a 500 error, and I’m unable to make any successful requests.

Used the open ai compatible endpoint https://preview.snova.ai/v1

And the endpoint is incompatible on open webui

Could you please look into this and provide assistance?

Best regards,

[coby.adams]

@abkjay

If you are entering a base url you should not have the /v1

Try this as a test to see if it works outside of cline and try cline with either the full chat completions or without the /v1

curl -H "Authorization: Bearer "
-H “Content-Type: application/json”
-d ‘{
“stream”: true,
“model”: “DeepSeek-R1”,
“messages”: [
{
“role”: “user”,
“content”: “Hello”
}
]
}’
-X POST https://preview.snova.ai/v1/chat/completions

-Coby

[abkjay]

I get 404 page not found with full chat completions and without v1 as well

Sharing the logs as well from docker for openwebui client

‘Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8’, url=‘https://preview.snova.ai/models’
2025-03-01 19:04:39 Traceback (most recent call last):
2025-03-01 19:04:39 File “/app/backend/open_webui/routers/openai.py”, line 523, in verify_connection
2025-03-01 19:04:39 res = await r.json()
2025-03-01 19:04:39 ^^^^^^^^^^^^^^
2025-03-01 19:04:39 File “/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py”, line 1281, in json
2025-03-01 19:04:39 raise ContentTypeError(
2025-03-01 19:04:39 aiohttp.client_exceptions.ContentTypeError: 400, message=‘Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8’, url=‘https://preview.snova.ai/models’
2025-03-01 19:04:39 INFO: 172.18.0.1:61578 - “POST /openai/verify HTTP/1.1” 500 Internal Server Error

[seth.kneeland]

Hey Abkjay,
Can you repoint to the api.sambanova endpoint and test again?

import os
import openai

client = openai.OpenAI(
    api_key=os.environ.get("SAMBANOVA_API_KEY"),
    base_url="https://api.sambanova.ai/v1",
)

response = client.chat.completions.create(
    model="DeepSeek-R1",
    messages=[{"role":"system","content":"You are a helpful assistant"},{"role":"user","content":"Hello"}],
    temperature=0.1,
    top_p=0.1
)

print(response.choices[0].message.content)

[abkjay]

Tested again. Getting the same error ERROR [open_webui.routers.openai] Client error: 400, message=‘Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8’, url=‘https://api.sambanova.ai/v1/models’
2025-03-01 21:45:48 Traceback (most recent call last):
2025-03-01 21:45:48 File “/app/backend/open_webui/routers/openai.py”, line 523, in verify_connection
2025-03-01 21:45:48 res = await r.json()
2025-03-01 21:45:48 ^^^^^^^^^^^^^^
2025-03-01 21:45:48 File “/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py”, line 1281, in json
2025-03-01 21:45:48 raise ContentTypeError(
2025-03-01 21:45:48 aiohttp.client_exceptions.ContentTypeError: 400, message=‘Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8’, url=‘https://api.sambanova.ai/v1/models’
2025-03-01 21:45:48 INFO: 172.18.0.1:59900 - “POST /openai/verify HTTP/1.1” 500 Internal Server Error
2025-03-01 21:45:55 ERROR [open_webui.routers.openai] Client error: 400, message=‘Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8’, url=‘https://api.sambanova.ai/v1/models’
2025-03-01 21:45:55 Traceback (most recent call last):

[seth.kneeland]

Hey Abkjay

I tried the cline plugin - I showed the following warning there:

(Note: Cline uses complex prompts and works best with Claude models. Less capable models may not work as expected.)

capable schmaple.
I use the continue plugin that might work as an alternative.
Here’s a link if you want to try:

SambaNova Cloud | Continue

[seth.kneeland]

Hey Abkjay,

I was able to get llama3.3 70b working with cline.
I’ll dig around a bit more and see if I can sort out r1.

Thanks

Seth

[seth.kneeland]

Got r1 working in webui locally.
Our api doesn’t have a model list yet, I had to configure a connection and model names by hand:

[seth.kneeland]

@abkjay Can you try the direct connection method and let me know if this helps? I think its the get model names routine that is breaking things.
Thanks!
Seth

[febryanvaldo089]

SambaNova doesn’t support /models endpoints, so you have to add the models manually to the OpenWebUI. I could make it work on OWUI.

[abkjay]

Sure thank you. I got it wirking on owui. But What about cline?

[schaveyt]

There is a github ticket on Cline with a response from the founder
Requested generation length 1 is not possible! · Issue #294 · cline/cline

There seems to be something on SambaNova side

I would strongly encourage the SambaNova team to get their API operational with Cline (it will then work with RooCode as well) and you will definitely see a up tick in API purchases

Even more if you contributed to Cline a SambaNova provider!!!