Context Length for the Meta-Llama-3.1-405B-Instruct is too small

[coby.adams]

@nikhilswami1 @stu and @daslav.rm

I know it is not 100 percent what you are looking for but we have expanded context lengths thus far on Llama 3.1 8 and 70b. As we expand more we will announce it in the release notes as we push to production.

Release Notes

Stay updated on the latest developments of the SambaNova Cloud! October 10, 2024 Llama 3.1 8B model: max sequence length increased from 8k to 16k. Llama 3.1 70B model: max sequence length increased from 8k to 64k. Automatic Routing Based on Sequence Length: You no longer need to change the model name to specify different sequence lengths. The system will automatically route requests based on sequence length. For example, there is no need to use Meta-Llama-3.1-70B-Instruct-8k for the 8k sequen…

-Coby