Are we moving higher than 32k ? needed for detailed workloads
1 Like
Hi david,
Thanks for checking! We’ll check with our internal team regarding support beyond 32k context and get back to you.
Hi david,
Yes, we’re moving beyond 32k. I checked internally, and the team confirmed that the context length will be increased.
Thank you.
2 Likes
@david.keane This is really good feedback for us. Yes we will work hard to move beyond 32k
Just out of curiosity with longer context windows are you running into Needle in the Haystack issues? Some of our other customers have reported seeing that.
I know it depends on workload but we also want to balance speed and accuracy.
2 Likes
Mostly just to allow the context to contain a larger result set and then process bigger data sets - we have it workig with 32k but its just limiting
1 Like