The electric network frequency (ENF) is recorded in the videos taken under the lights powered by grid and can be used for digital forensics. However, due to the lack of data caused by the low frame rate of the video, the ENF-based forensics methods always need a reference signal extracted from the grid, which limits the practical application of these methods. In this article, a new ENF-based time domain video forgery detection algorithm is proposed to solve the problem of data lack. The cubic spline interpolation is used to generate suitable data points of the ENF signal, and the detection sequence generated based on the correlation coefficient between data points in adjacent periods is used to catch the phase continuity interruption of the ENF signal and detect the exact position of forgery. The proposed algorithm can be used independently without any reference signals. The experimental results show that the proposed algorithm has good performance in detecting forgery videos with varying degrees of deletion, duplication and insertion of frames.