Despite the simplicity of the whale optimization algorithm (WOA) and its success in solving some optimization problems, it faces many issues. Thus, WOA has attracted scholars' attention, and researchers frequently prefer to employ and improve it to address real-world application optimization problems. As a result, many WOA variations have been developed, usually using two main approaches improvement and hybridization. However, no comprehensive study critically reviews and analyzes WOA and its variants to find effective techniques and algorithms and develop more successful variants. Therefore, in this paper, first, the WOA is critically analyzed, then the last 5 years' developments of WOA are systematically reviewed. To do this, a new adapted PRISMA methodology is introduced to select eligible papers, including three main stages: identification, evaluation, and reporting. The evaluation stage was improved using three screening steps and strict inclusion criteria to select a reasonable number of eligible papers. Ultimately, 59 improved WOA and 57 hybrid WOA variants published by reputable publishers, including Springer, Elsevier, and IEEE, were selected as eligible papers. Effective techniques for improving and successful algorithms for hybridizing eligible WOA variants are described. The eligible WOA are reviewed in continuous, binary, single-objective, and multi/many-objective categories. The distribution of eligible WOA variants regarding their publisher, journal, application, and authors' country was visualized. It is also concluded that most papers in this area lack a comprehensive comparison with previous WOA variants and are usually compared only with other algorithms. Finally, some future directions are suggested.