Maximum likelihood learning of RBMs with Gaussian visible units on the Stiefel manifold

The restricted Boltzmann machine (RBM) is a generative model widely used as an essential component of deep networks. However, it is hard to train RBMs by using maximum likelihood (ML) learning because many iterations of Gibbs sampling take too much computational time. In this study, we reveal that, if we consider RBMs with Gaussian visible units and constrain the weight matrix to the Stiefel manifold, we can easily compute analytical values of the likelihood and its gradients. The proposed algorithm on the Stiefel manifold achieves comparable performance to the standard learning algorithm.