Is Llama 3.1 405B Temporarily Unavailable?

SambaNova Devs
1

I encountered this error while using the API. Other models are working normally:

{'error': {'code': None, 'message': 'Meta-Llama-3.1-405B-Instruct is temporarily unavailable. Please try again later!', 'param': None, 'type': ''}}

1 Like
2

@pcchen It seems to be working for me are you still having issues?

-Coby

3

It seems I can use it now, but the speed is slower. It was unavailable for about 2 hours before.

However, after a while, the message “Meta-Llama-3.1-405B-Instruct is temporarily unavailable. Please try again later!” appeared again.

4

This is your requests per minute.

You are on the free tier and are frequently going beyond the free tier rate limit for 405b which is 7 a minute. All calls outside that range will get you an unavailable message the error code is a 429

undefined
undefined1148Ă—270 21.8 KB

-Coby

1 Like
5

Thank you for your explanation. I understand now.

6

@pcchen My pleasure. Our self service developer tier subscription mechanism is on the near horizon to purchase higher rate limits. I do see you are in a university. Can you provide a bit of information on what you are working on and what rate limits you would need?

-Coby

7

I am conducting NLP research on the impact of different prompting strategies on the responses of large language models (LLMs).
Will the free plan remain available in the future?

8

@coby.adams
I would like to ask, although I am not using a high rate with Llama 3.1 405B, I keep seeing the message “eta-Llama-3.1-405B-Instruct is temporarily unavailable. Please try again later!” quite often. ould there be some other issue?

9

@coby.adams
Your api and site are crashing.
Unable to even login
Might be the effect of your speec at davos 2 hours back.

10

@kollikiran456 please deifne crashng? What errors are you seeing I am looking at our monitoring and I do not see these issues in my graphs.

-Coby

11

I confirm - this model does not work in the free plan, even on a new account. It works through playground, but not through api

12

Hi All,

To provide an update to this thread we have an ongoing issue with our 405b model, latest updates can be seen here: SamabNova Cloud Status - Meta-Llama-3.1-405B-Instruct giving intermittent unavailable

Thanks & Regards
Alex

1 Like
13

All we have identified the issue and put in a preventative. We will continue to monitor.
-Coby

2 Likes
14

It is again showing
{"error": {"code": None, "message": "Meta-Llama-3.1-405B-Instruct is temporarily unavailable. Please try again later!", "param": None, "type": ""}},
but the SamabNova Cloud Status does not indicate any errors.

Is there currently an issue with the SamabNova Cloud API?

1 Like
15

@pcchen We are investigating
-Coby

1 Like
16

@pcchen It should be better now . Please test when you can.

-Coby

17

It’s working now. Thanks!

1 Like