Ranking for Coding?

Hi! Any ranking for the models in SambaNova for using at coding? Thanks!

Hi @resoluciones ,

Thank you for your question! There isn’t an official ranking for models in SambaNova specifically for coding. However, many users prefer a combination approach: using Llama 3.3 for general long-context analysis and then refining shorter contexts with models like QWQ, DeepSeek, or Qwen Coder. This blend helps achieve both broad understanding and precise code generation. Let us know if you need more details.

Regards,
Durgesh

1 Like

In my own testing I found DeepSeek-R1 and DeepSeek-V3-0324 strongest.

Qwen2.5-Coder-32B-Instruct is a very strong coding model, especially for its size.

Meta-Llama-3.3-70B-Instruct and Meta-Llama-3.1-405B-Instruct are good at conversations where code needs to be explained (whereas Qwen2.5-Coder-32B-Instruct is better at providing the actual code)

Here are my rankings in a table:

Model Name Strengths
DeepSeek-R1 Strongest overall
DeepSeek-V3-0324 Strongest overall
Qwen2.5-Coder-32B-Instruct Strong coding model, especially for its size. Better at providing actual code
Meta-Llama-3.1-405B-Instruct Good at conversations where code needs to be explained
Meta-Llama-3.3-70B-Instruct Good at conversations where code needs to be explained
1 Like

Wow, great breakdown! @fox I’m curious which coding AI tools are you using?
Or are you just making calls to the cloud?

1 Like

Hi @justin.woo !

Thank you! At the moment I’m not using any AI coding tools. Like you mentioned, it’s calls to cloud models, usually with a few back-and-forth’s to get the desired result. By doing this, I feel like I got somewhat of an idea of the strengths of each model.

I did look into aider (https://aider.chat) and that looks really promising! I might give that a try soon.

It’s interesting seeing people make entire projects with those tools, and even revise them later on by importing the project. It is my understand that the importing is sometimes done behind the scenes by simply including the repo code in the request (so technically through the prompt itself) or with some kind of (vector?) database. Maybe even RAG into a repo.

As you can probably tell, I’m still learning the tools part of it all :slight_smile:.

1 Like

Hi @fox!

Thanks for the detailed reply. Aider looks amazing I have to try it out!
So far here are some of the tools I have been looking into that allow you to use open source models with your own provider and also use the IDE to include your repo code.

As you explore more I’m excited to learn from you and the tools you are using. This space is moving so fast and I feel every other day there is a new tool coming out!

2 Likes

Awesome @justin.woo , I look forward to learning from you too! Those are some great tools, I will have to try them out for sure. And I totally agree, this space moves so fast!

DeepSeek-R1 + DeepSeek-V3-0324 is the best partner, but the context is too short.

I totally feel the too. Been trying out Sambanova for some small dev one offs (and enjoy the speed) but the context of Deepseek’s models are restricting me to testing with only aider. Hope the context increases as it gets out of beta!