Heart rate (HR) signal analysis is widely used in the medicine and medical research area. Physical activities (PA) are commonly recognized to greatly affect the changes of heart rate. A method of Evolutionary Neural Network -- Neuroevolution of Augmenting Topologies (NEAT) is used to build a PA-based HR predictor model. Through special coding, crossover and mutation operator, NEAT can implement network topology and connectivity weights evolution simultaneously. The common problem in evolutionary neural network, like competing conventions, how to protect the new innovation are effectively solved. The experimental results demonstrated the application potential of the approach.