The Arithmetic Optimization Algorithm (AOA) is a recently proposed metaheuristic algorithm that has been shown to perform well in several benchmark tests. The AOA is a metaheuristic that uses the main arithmetic operators’ distribution behavior, such as multiplication, division, subtraction, and addition. This paper proposes a binary version of the Arithmetic Optimization Algorithm (BAOA) to tackle the feature selection problem in classification. The algorithm’s search space is converted from a continuous to a binary one using the sigmoid transfer function to meet the nature of the feature selection task. The classifier uses a method known as the wrapper-based approach K-Nearest Neighbors (KNN), to find the best possible solutions. This study uses 18 benchmark datasets from the University of California, Irvine (UCI) repository to evaluate the suggested binary algorithm’s performance. The results demonstrate that BAOA outperformed the Binary Dragonfly Algorithm (BDF), Binary Particle Swarm Optimization (BPSO), Binary Genetic Algorithm (BGA), and Binary Cat Swarm Optimization (BCAT) when various performance metrics were used, including classification accuracy, selected features as well as the best and worst optimum fitness values.