Non-iterative Learning Approaches and Their Applications

Optimization, which plays a central role in learning, has received considerable attention from academics, researchers, and domain workers [3]. Many optimization problems in machine learning can be tackled with noniterative approaches, which can be solved in closed-form manner [4]. Those methods are in general computationally faster than iterative solutions, such as the stochastic gradient descent, used in modern deep learning architectures [2]. Even though non-iterative methods, such as Echo State Networks [1], Extreme Learning Machines [5], and Random Vector Functional Link [6], have attracted much attention in recent years, there exists a performance gap when compared with older methods and other competing paradigms. The main goal of this special issue is to present recent advances in non-iterative solutions in learning that can reduce such a gap. Secondly, the special issue focuses on showing the important advantages of non-iterative optimization compared with the iterative counterpart, such as gradient-based methods and derivative-free iterative optimization techniques. Besides the dissemination of the latest research results on non-iterative algorithms, the special issue covers practical applications, presents new methodological paradigms, and identifies directions for future studies. We selected six papers to appear in this special issue. All of them have gone through at least two rounds of revision by two to four expert reviewers that have been carefully selected. One of the papers, coauthored by one of the guest editors, underwent an independent review process to guarantee fairness. The papers published in the special issue are summarized in the following.