Android devices are equipped with various sensors. Permissions from users must be explicitly granted for apps to obtain sensitive information, e.g., geographic location. However, some of the sensors are considered trivial such that no permission control is enforced over them, e.g., the ambient light sensor. In this work, we present a novel side channel, i.e. the ambient light sensor, that can be used to track the mobile users. We develop a location tracking system with off-line trained route identification models using the values from the attacker’s own ambient light sensor. The system can then be used to track a user’s geographic location. The experiment results show that our route identification models achieve a high accuracy of over 91% in user’s route identification and our tracking system achieves an accuracy at about 64% in real-time tracking the user with estimation error at about 70 m. Our system out-performs the state-of-the-art works with other side channels. Our work shows that with merely the values from the ambient light sensor of user’s mobile phone that requires zero-permission to access, the geographic routes that the users have taken and their real-time locations can be identified with machine learning techniques in high accuracy.