Private LLM Inference using Fully Homomorphic Encryption
I gave this talk as the second webinar in the monthly OpenFHE series. I presented a conceptual summary of our comprehensive study of private Large Language Model (LLM) inference using Fully Homomorphic Encryption (FHE), which surveys all known frameworks developed in this area over the past 2–3 years. The webinar ran for about 111 minutes, and the audience was highly engaged: the Q&A session was as long as the main presentation, with 22 questions on both the results and the future outlook for FHE-based private LLM inference.
In the talk, I answered three main questions:
- Can FHE achieve practical accuracy in evaluating LLMs?
- What approaches can be used for developing production-ready solutions?
- What is the current overhead of FHE compared to cleartext LLM implementations?
I also presented POLARIS, our own framework built on OpenFHE and FIDESlib that implements the best practices developed in our paper.
Abstract
This is the second webinar in the monthly series presenting major projects related to the OpenFHE library. Private LLM inference using FHE has been an active area of research, with many frameworks developed in the past 2–3 years. We just completed a comprehensive study examining all known private LLM inference frameworks. This talk presents a conceptual summary of that study and introduces POLARIS, our framework based on OpenFHE and FIDESlib.