Request for Assistance in Integrating Hugging Face Model with FastAPI

SambaNova Devs Discussion
1

Hi,

I’m currently exploring the “Getting Started” section and am interested in
integrating the following Hugging Face model into a FastAPI application:
Llama-3.1-8B-Lexi-Uncensored.
I’ve read that if my custom checkpoint is supported, I can bring it into
FastAPI by providing the necessary safetensors and configurations.

Could you please guide me on how to onboard this checkpoint and proceed
with the integration?

2 Likes
2

This feature is not available as of today but it is on the near term road map.

3

please express interest in this forum.

1 Like