Traditional color trackers tend to fail in some challenging scenarios such as object deformation, rotation, background clutter and varying illumination. In this paper, we take advantages of the material identification ability of hyperspectral images to tackle the object tracking problem. Abundances and spectral-spatial histogram of oriented gradients (SSHOG) are adopted as material features for tracking. Abundances extract the material distribution by performing dynamic joint online unmixing, in which the temporal information is used to suppress the effect of spectral variability between adjacent frames. SSHOG summarizes the local spectral-spatial oriented gradients to describe the local 3D textures of the target. These features are further embedded to correlation filters, yielding a novel dynamic material-aware tracking (DMT) method. Experimental results on hyperspectral benchmark show the superiority of DMT over other trackers.