Counterexample-Guided Learning of Monotonic Neural Networks
34th Conference on Neural Information Processing Systems (NeurIPS 2020), December 6-12, 2020.
Aishwarya Sivaraman, Golnoosh Farnadi, Todd Millstein, Guy Van den Broeck
The widespread adoption of deep learning is often attributed to its
automatic feature construction with minimal inductive bias. However, in many real-world tasks,
the learned function is intended to satisfy domain-specific constraints. We focus on monotonicity
constraints, which are common and require that the function's output increases with increasing
values of specific input features. We develop a counterexample-guided technique to provably
enforce monotonicity constraints at prediction time. Additionally, we propose a technique to use
monotonicity as an inductive bias for deep learning. It works by iteratively incorporating
monotonicity counterexamples in the learning process. Contrary to prior work in monotonic
learning, we target general ReLU neural networks and do not further restrict the hypothesis
space. We have implemented these techniques in a tool called COMET. Experiments on real-world
datasets demonstrate that our approach achieves state-of-the-art results compared to existing
monotonic learners, and can improve the model quality compared to those that were trained
without taking monotonicity constraints into account.
[PDF | Implementation]