Deep neural networks (DNNs) have been frequently and widely applied for intelligent systems such as object detection, natural language understanding and speech recognition. Given a specific problem, we always aim to construct the most suitable DNN to solve it, which requires choosing the most appropriate model architecture and seeking the best model parameters values. However, most existing works focus on model parameters learning under the assumption that the model architecture can be manually specified as per prior knowledge and/or trial-and-error experimentation. To overcome this problem, evolutionary algorithms (EAs) have been widely used to design model architectures automatically. Further, EAs have been used for neural network optimization for more than 30 years. Therefore, in this paper, we review the evolutionary neural architecture search (ENAS) from the view of the advanced techniques. We hope this work can provide a comprehensive understanding of EAs’ roles for the readers and focus themselves on ENAS.