FHE in AI workflow

SambaNova Devs Discussion
1

Does anyone seen a flow where user encrypt the prompt with FHE then the model process it in the encrypted form, and returns the output also in the encrypted form?

I think such flow would be the ideal for future models.

1 Like
2

@hello1 you would have to fine tune the model with encrypted data to facilitate such things. I have seen in AWS sagemaker where folks have experimented with this but there does not appear to be wide adoption yet.
-Coby

1 Like
3

Thanks Coby, I’m already started to write an example code. I will do it with a GPT-2 model as an example to see how it works.

Andrej Karpathy’s GPT2 c code is a good start for this.

1 Like
4

I found this blog, and it shows I’m not the only one who is interested:

I will try to assemble on my research day, but my question is more on the SambaNova hardware infrastructure, how it capable to accelerate such workloads.
Maybe an engineer can explain what is possible, or an intern who has access to hardware can benchmark the code.

I think it would be interesting for inference providers, as technology provides the guarantee for the data privacy.