Visualizing Optimizers using Chebyshev Proxies and Fatou Sets
Visualizing the behaviour of numerical optimizers by treating them as dynamical systems and rendering their Fatou sets.
Optimizers are usually judged by convergence curves, which say little about how they explore a landscape. By approximating the objective with Chebyshev proxies and rendering the resulting Fatou sets, the dynamics of an optimizer — its basins, its boundaries, its instabilities — become directly visible.
With recent advances in optimization many different optimization approaches have been proposed, especially regarding the optimization of weights for neural networks. However, comparing these approaches in a visually succinct and intuitive manner is difficult to do, especially without relying on simplified toy examples that may not be representative. In this paper, we present a visualization toolkit using a modified variant of Fatou sets of functions in the complex domain to directly visualize the convergence behavior of an optimizer across a large range of input values. Furthermore, we propose an approach of generating test functions based on polynomial Chebyshev proxies, with polynomial degrees up to 11217, and a modification of these proxies to yield functions that are strictly positive with known global minima, i.e., roots. Our proposed toolkit is provided as a cross platform open source framework in C++ using OpenMP for parallelization. Finally, for menomorphic functions the process generates visually interesting fractals, which might also be interesting from an artistic standpoint.
@inproceedings{DBLP:conf/vmv/WinchenbachT22,author={Winchenbach, Rene and Thuerey, Nils},title={Visualizing Optimizers using Chebyshev Proxies and Fatou Sets},booktitle={{VMV} 2022, 27th International Symposium on Vision, Modeling, and
Visualization, Konstanz, Germany, September 27-30, 2022},pages={75--82},publisher={Eurographics Association},year={2022},doi={10.2312/vmv.20221206},url={https://doi.org/10.2312/vmv.20221206},}
-1400.webp)