[FeatureRequest] - Endpoint without error 429

SambaNova Devs Discussion
1

I know the reason behind the rate limits, but it would be great to have an endpoint that limits without error, just waiting until the next request happens.

An async framework, such as Nginx, differs from Apache and can be archived without extra error handling on the application side, especially if the API is used for many components, such as a multi-agent application.

If there is another way to handle it on the client side that is best, please let me know.

Regarding speed, most of these calls are not performance-critical, but I think the 429 error calls also log overhead on your side.

Thanks,

Laszlo

1 Like
2

One update - I finally made a custom LLM handler, which uses the defined rate limits, this way you can reduce the priority for the request.

1 Like
3

Hello hello1!

Glad you were able to craft a suitable workaround. While we are improving our features, would you be willing to post a snippet for others to use?

Keep those feature requests coming, they help us prioritize our efforts.

Thanks for being an active member of our community!

Seth

4

Sure, over the weekend, I will create a GitHub repository for the code.

I have some code parts which more related to my project, I have to remove it.

1 Like
5

Hi Seth,

Below is my repository address, where I shared a simplified code that can be used for multiple purposes.

Anyone can use the code freely.

https://github.com/cuttie1979/sambanova_factory

With Best Regards,

     Laszlo
2 Likes