Mental illnesses are becoming increasingly prevalent, in turn leading to an increased interest in exploring artificial intelligence (AI) solutions to facilitate and enhance healthcare processes ranging from diagnosis to monitoring and treatment. In contrast to application areas where black box systems may be acceptable, explainability in healthcare applications is essential, especially in the case of diagnosing complex and sensitive mental health issues. In this paper, we first summarize recent developments in AI research for mental health, followed by an overview of approaches to explainable AI and their potential benefits in healthcare settings. We then present a recent case study of applying explainable AI for ADHD diagnosis which is used as a basis to identify challenges in realizing explainable AI solutions for mental health diagnosis and potential future research directions to address these challenges.