The development of effective prediction models using automatic sleep stage classification (ASSC) has accelerated significantly in recent years, reducing clinicians' workload in sleep investigations. Deep learning methods, coupled with cutting-edge ideas such as attention mechanisms and transfer learning, are gaining popularity alongside conventional machine learning techniques. In this study, we provided a comprehensive summary of research published in the last 5years (since 2017) in six categories and identified critical open issues in ASSC development. These include training models on small datasets, developing personalized models, creating interpretable models, enabling long-term monitoring, generalization source, reducing bias, and quantifying uncertainty through recursive upgrades of ASSC models. The findings of the review are crucial to advancing current research in ASSC and developing more effective and user-friendly sleep monitoring tools that will be widely accepted in healthcare. Addressing the identified open issues will significantly improve the accuracy and usefulness of ASSC models, resulting in better sleep diagnostic, monitoring, and treatment options for individuals with sleep disorders.