Ensuring health systems responsiveness is crucial for health equity and outcomes of all individuals, particularly disadvantaged groups such as people with disabilities. However, attention to and discussions on health system responsiveness for people with disabilities remains lacking. This viewpoint highlights the pervasive issues within health systems rooted in ableism and proposes an agenda to tackle ableism, aiming to make health systems responsive to the needs of people with disabilities. Their needs are complex and diverse, varying with the disability, its severity, progression, and intersection with other factors. Ableism creates significant obstacles to identifying and addressing their needs and expectations, damages provider–patient interactions, poses multiple challenges in healthcare, and impacts the overall responsiveness of the health system to the populations it is meant to serve. The proposed agenda outlines areas for action and research across six building blocks of health systems as a way forward to enhance the health system's responsiveness to the needs of people with disabilities.