The paper Convexity in ReLU Neural Networks: beyond ICNNs? by Anne Gagneux, Mathurin Massias, Emmanuel Soubies and Remi Gribonval, was accepted for publication at the Journal of Mathematical Image and Vision. https://arxiv.org/abs/2501.03017
In this paper, we develop a toolkit to characterize neural networks implementing convex functions, which play a prominent role in Optimal Transport and Plug and Play methods in imaging. We show that the standard architecture to implement such functions, the so-called Input Convex Neural Networs, are too restrictive : there exist many networks implementing convex functions that are not ICNN. We even show that there are convex functions implemneted by a given network which cannot be implemented by an ICNN with the same architecture.
