Building `livekit-plugins-sambanova` - native STT support for LiveKit Agents

SambaNova Devs Discussion
1

Hey everyone! :waving_hand:

I’ve been building voice AI agents with LiveKit and SambaNova Cloud, and I noticed there’s no dedicated LiveKit plugin for SambaNova yet.

Current workaround from SambaNova’s own LiveKit docs:

stt = openai.STT(
    model="Whisper-Large-v3",
    api_key=SAMBANOVA_API_KEY,
    base_url="https://api.sambanova.ai/v1",
)

This works, but providers like Groq, Deepgram, and Cerebras all have dedicated plugins — livekit-plugins-groq, livekit-plugins-deepgram, etc. — with proper model enums, env var handling, and first-class documentation on the LiveKit integrations page.

What I want to build: A standalone livekit-plugins-sambanova package starting with STT (Whisper-Large-v3), following the same plugin architecture. This would give developers:

from livekit.plugins import sambanova

stt = sambanova.STT()  # reads SAMBANOVA_API_KEY, defaults to Whisper-Large-v3

SambaNova’s 245x Whisper speed factor is a strong differentiator for real-time voice agents, it deserves proper plugin support, not a base_url workaround.

I’ll be filing the issue on the github - livekit/agents shortly. Would love to know:

  1. Any upcoming changes to the Whisper endpoint I should plan around?

  2. Would anyone from the team be open to reviewing/endorsing the PR?

Happy to share progress here as it develops.

2 Likes
2

This is a great call. SambaNova’s Whisper speed makes it a serious contender for real-time voice agents — a dedicated LiveKit plugin would absolutely elevate the integration beyond the OpenAI-compatible workaround.

I completely agree with your approach of following the existing plugin architecture for consistency and ease of use.

It’s always a good idea to keep an eye on the SambaNova API documentation for any upcoming changes to the Whisper endpoint. If needed, you can also reach out to our support team for additional insights.

While I’ll check do we have team for dedicated exclusively to PR reviews, but now our engineering and supporting team is always open to collaborating with the community.

Here’s how we can support:

• Technical Guidance – Answer questions around the SambaNova API or Whisper integration
• Testing & Feedback – Help validate functionality and performance
• Documentation Review – Ensure clarity and accuracy for developers

Please feel free to reach out here in the community if you need assistance. We’re looking forward to seeing this plugin come to life.

Would love to see this move forward — happy to help test.

2 Likes
3

Thank you for the support and the offer to collaborate, that’s exactly what I was hoping for! :raising_hands:

One quick question, is there a preferred way to reach the engineering team for API-specific questions as they come up? Happy to keep it in this thread, or if there’s a Discord/Slack I should join, let me know.

Will update this thread with progress. Let’s make this happen! :rocket:

Thanks!

attaching issue from gh for ref: livekit/agents/issues/4900

2 Likes
4

Thank you for the support and the willingness to collaborate — really appreciate it.

I’ve created a Product Feedback request to track this internally. I’ll continue coordinating through that channel as API-specific questions come up.

For reference, I’ve also attached the related GitHub issue here:
livekit/agents/issues/4900

I’ll keep the thread updated with progress as development moves forward.

Thanks again — looking forward to making this happen.

Best regards,
Hitesh

2 Likes
5

Thanks Hitesh,

I started working on the STT Integration, - livekit/agents/pull/4902 will update here, as progress

2 Likes
6

Hi @hitesh.jha, can you please help me to check

I have been getting 404, on using Whisper - I tried from curl, sambanova sdk and openai sdk. Here is the base script failing:

from sambanova import SambaNova
import base64

client = SambaNova(
    base_url="your-sambanova-base-url",
    api_key="your-sambanova-api-key",
)

audio_path="sample.wav"
with open(audio_path, "rb") as audio_file: 
   bin_audio = audio_file.read()

response = client.audio.transcriptions.create(
    model="Whisper-Large-v3",
    file=(audio_path,bin_audio),
)
print(str(response))

Traceback:

sambanova.BadRequestError: invalid or corrupted audio file(s): sample.wav

openai.BadRequestError: invalid or corrupted audio file(s): sample.wav

Test on Python 3.10 and Python 3.13
Sample Audio - 16-bit PCM, mono, 44.1kHz, 9.74s,

Find the exact code and audio file - github com/mahimairaja/test-sambanova-whisper

1 Like
7

Thank you for reaching out here. I will create a support case for the reported issue, you will receive a confirmation email shortly. I will escalate this to the appropriate team so they can review/reproduce and investigate the same. We will keep you updated as we learn more.

2 Likes
8

Thanks Raheem, I am looking forward

9

Meantime, I started implementing the LLM func for the plugin here - github com/livekit/agents/pull/4910

1 Like
10

Hi Team, the first PR for LLM integration for the livekit plugin is ready.

github com/livekit/agents/pull/4910

Can anyone please review and endorse, thanks

1 Like
11

Thanks @mahimairaja3 for sharing the details — I dug into this and here’s what I’ve confirmed so far.

What we verified:

  • The audio file is valid (PCM s16le, mono, 16 kHz).

  • File size and WAV container look correct.

  • The model Whisper-Large-v3 is present in the /v1/models list for this tenant.

  • The issue reproduces across both the SambaNova SDK and the OpenAI SDK.

Given all of the above, this does not appear to be an audio formatting issue. The remaining likely causes are:

  • Multipart/form-data handling differences between the SDK and the SambaNova gateway

  • A backend validation quirk specific to the Whisper endpoint

  • Or a tenant-level configuration issue despite the model being listed

Suggested next steps for isolation

  1. Test the same request via raw curl:

    curl https://api.sambanova.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer <API_KEY>" \
  -F file=@sample.wav \
  -F model=Whisper-Large-v3

  2. If curl also fails, this strongly points to a backend or account configuration issue.

  3. If curl succeeds, the problem is likely SDK multipart encoding.

At this point, we will check with the SambaNova internal teams and will confirm whether there are any known issues or additional requirements for the Whisper transcription endpoint on this tenant.

Happy to provide further logs if needed.

1 Like
12

Thanks Hitesh, I tried with curl. I also failed.

curl https://api.sambanova.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer <API_KEY>" \
  -F file=@sample.wav \
  -F model=Whisper-Large-v3

invalid or corrupted audio file(s): sample.wav

Kindly, help me to debug further. Thanks

And a quick note. I have raise PR for the LLM part and got confirmation from livekit team.

here: github com/livekit/agents/pull/4910

It would be great if anyone from SambaNova endorse the PR. Thanks

1 Like
13

Thanks for the clarification.

Understood this falls under SambaNova Engineer Team. Where our goal with the PR is to add native plugin support (beyond the current workaround) for a cleaner LiveKit integration. Happy to align with any preferred patterns from the SambaNova side.

Separately, regarding the invalid or corrupted audio file(s): sample.wav error — I’ll continue debugging the audio formatting on my end, but please let me know if there are any specific codec/container requirements I should validate against.

Would appreciate any feedback on the PR whenever the team gets a chance , Team will do the endorsement as they get chance to review. Thanks for the support!

1 Like
14

Thank you so much Hitesh. The PR for LLM just got merged. github com/livekit/agents/pull/4910

I will stay on the loop untill we figure out the whisper. Thank yoU!

1 Like