Location estimation or localization is one of the key components in IoT applications such as remote health monitoring and smart homes. Amongst device-free localization technologies, passive infrared (PIR) sensors are one of the promising options due to their low cost, low energy consumption, and good accuracy. However, most of the existing systems are complexly designed and difficult to deploy in real life, in addition, there is no public dataset available for researchers to benchmark their proposed localization and tracking methods. In this paper, we propose a system and a dataset collected from our PIR system consisting of commercial-of-the-shelf (COTS) sensors without any modification. Our dataset includes profile data of 36 classes that have over 1,000 samples of different walking directions and test data consisting of multiple scenarios with a sequence length of over 2,000 timesteps. To evaluate our system and dataset, we implement various deep learning methods such as CNN, RNN, and CNN–RNN. Our results prove the applicability and feasibility of our system and illustrate the viability of deep learning methods for PIR-based localization and tracking. We also show that our dataset can be converted for coordinate estimation so that deep learning methods and particle filter approaches can be applied to estimate coordinates. As a result, the best performer achieves a distance error of 0.25 m.