Evolutionary Algorithms (EAs) are powerful tools for searching the large and intricate search spaces found in difficult problems. Despite their effectiveness, EAs are computationally expensive, often prematurely converge to local optima, and have practical limitations due to prolonged execution times. The focus of this thesis is to exploit NVIDIA Graphics Processing Unit (GPU) hardware to accelerate and enhance the performance of EAs. To achieve this a deep understanding of the NVIDIA GPU architecture is required and new ideas must be incorporated into the EAs to successfully map them to this hardware. To gain a deeper understanding of EA dynamics, this work introduces a novel visualisation tool designed to provide insights into EAs applied to numerical optimisation. Following this, a state-of-the-art GPU-accelerated Island Model Genetic Algorithm (IMGA) was developed, achieving speedups of up to 36 times faster than optimised, multi-threaded Central Processing Unit (CPU)-based approaches. This development established the foundation for advancing GPU-accelerated island model EAs. Next, a GPU-based numerical optimisation benchmark was devised and used to compare existing GPU-based Differential Evolution (DE) algorithms. The GPU-accelerated island model framework was then combined with DE, showing improved convergence properties and wall-clock speeds when compared to existing GPU-based DE algorithms, providing substantial speedups of up to 470 times faster than a single-threaded and 95 times faster than multi-threaded CPU-based DE. Building on these results, two GPU-based Island Model DE (IMDE) variants were introduced, incorporating self-adaptive control parameters and dynamic search strategy selection through online learning, displaying superior convergence properties. This comprehensive work contributes to advancing the field of evolutionary computation, offering accelerated, efficient, and adaptive algorithms, while also providing a valuable visualisation tool to further evolutionary computation research.