Boolformer: Symbolic Regression of Logic Functions with Transformers

TL;DR — We introduce Boolformer, a Transformer model capable of inferring Boolean functions in symbolic form with state-of-the-art performance. We present applications to real-world binary classification tasks and inference of gene regulatory networks.

Abstract

In this work, we introduce the Boolformer, the first Transformer architecture trained to perform end-to-end symbolic regression of Boolean functions. First, we show that it can predict compact formulas for complex functions which were not seen during training, when provided a clean truth table. Then, we demonstrate its ability to find approximate expressions when provided incomplete and noisy observations. We compare it with classic machine learning approaches on a broad set of real-world binary classification datasets, demonstrating its potential as an interpretable alternative. Finally, we apply it to the widespread task of modelling the dynamics of gene regulatory networks. Using a recent benchmark, we show that Boolformer is competitive with state-of-the art genetic algorithms with a speedup of several orders of magnitude.