Kunze, Julius Kirsch, Louis Ritter, Hippolyt Barber, David 2019-08-03 Variational inference with a factorized Gaussian posterior estimate is a widely-used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show how mean field inference improves generalization by limiting mutual information between learned parameters and the data through noise. We quantify a maximum capacity when the posterior variance is either fixed or learned and connect it to generalization error, even when the KL-divergence in the objective is scaled by a constant. Our experiments suggest that bounding information between parameters and data effectively regularizes neural networks on both supervised and unsupervised tasks. application/pdf https://localhost:5000/ark:/12658/srd1318902 https://susi.usi.ch/global/documents/318902 https://susi.usi.ch/documents/318902/files/Kunze_E_2019.pdf eng info:eu-repo/semantics/altIdentifier/doi/10.3390/e21080758 info:eu-repo/semantics/altIdentifier/ark/12658/srd1318902 info:eu-repo/semantics/openAccess CC BY Entropy. - 2019, vol. 21, no. 8, p. 758 Information theory Variational inference Machine learning info:eu-repo/classification/udc/004 Gaussian mean field regularizes by limiting learned information http://purl.org/coar/resource_type/c_6501