Ventricular Assist Devices (VADs) provide continuous mechanical circulatory support during in- and out-of-hospital care. However, limitations of the external wearable components influence patient quality of life. There is insufficient understanding of the issues with such components that combines a holistic viewpoint from both human factors and user (including patient and caregiver) experience perspectives. This paper comprehensively details the issues with VAD wearable systems and proposes a way for human-centered design to bridge the gap, addressing such issues synergistically. Through the review the authors investigated: the user issues caused by wearable components of VADs, and how human factors issues correlate to the VAD user experience. A Boolean search (“ventricular assist” AND “human factors” AND “experience”) for peer-reviewed studies published between 2008 and 2018 returned 338 titles, with 35 relevant studies selected using a PRISMA process for inclusion in cross-study analysis and synthesis. This paper provides design recommendations for the issues found in the literature. Four key focus areas to inform the future design of VAD wearable components were identified—‘Power Supply’, ‘Wearability and Travel Freedom’, ‘The Female Experience’ and ‘Intuitive Handling’. Using design to drive innovation could result in VAD wearable components which better meet or exceed users’ quality of life goals.