Could you please increase my rate limits or allow me to use the paid version? I would like to support as many users as possible for my web app. Is it possible to increase it to >= 1,000 RPM? If not, what is the limit I am allowed to receive?
I want to increase it before I do a major launch soon and have a lot of users try my website. It is a web app for using AI for the bible.
@wesley Which model(s) and context length(s) would you be targeting to hit with such a rate limit? How many approximate tokens would you be consuming daily. monthly, annually ? Would you have a maximum TTFT/TTFC requirement?
@coby.adams I would be targeting the “Meta-Llama-3.3-70B-Instruct” model. I only need about 10,000 context length maximum per request.
My maximum approximate tokens:
Daily Est: 40 million
Monthly Est: 196 million
Annual Est: 2,548 million
Maximum TTFT: 3 seconds
Maximum TTFC: 7 seconds
Note: If it becomes popular, I will likely need a lot more in the future.
Hi Wesley,
Thank you for sharing your metrics and model type. I would like to set up a call tomorrow or Thursday based on your availability to dive deeper and fulfill your request. You can email me separately at sam.ajithkumar@sambanova.ai
Looking forward to our discussion