FHEFusion: Enabling Operator Fusion in FHE Compilers for Depth-Efficient DNN Inference
Dec 3, 2025 • cmplr
Abstract
Operator fusion is essential for accelerating FHE- based DNN inference because it reduces multiplicative depth and, in turn, lowers the cost of ciphertext operations by keeping them at lower ciphertext levels. Existing approaches either rely on man- ual optimizations, which miss cross-operator opportunities, or on compiler pattern matching, which lacks generality. Standard DNN graphs omit FHE-specific behaviors, while fully lowering to primitive FHE operations introduces excessive granularity and obstructs effective optimization. We present FHEFUSION, a compiler framework for the CKKS scheme that enables fusion through a new IR. This IR preserves high-level DNN semantics while introducing FHE-aware opera- tors—masking and compaction (Strided Slice)—that are central to CKKS, thereby exposing broader fusion opportunities. Guided by algebraic rules and an FHE-aware cost model, FHEFUSION reduces multiplicative depth and identifies profitable fusions. Integrated into ANT-ACE, a state-of-the-art FHE compiler, FHEFUSION outperforms NGRAPH, the only framework with graph-level fusion, achieving up to 3.02× (average 1.40×) speedup across seven DNNs (13 variants from different RELU approximations) on CPUs, while maintaining inference accuracy.
IEEE Reference Format
T. Sui et al., “FHEFusion: Enabling Operator Fusion in FHE Compilers for Depth-Efficient DNN Inference,” in 2026 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), Sydney, Australia, 2026, pp. 70-83, doi: 10.1109/CGO68049.2026.11395213