The unbefitting binding energy for H*/OH* with Ni sites greatly restrains the electrochemical activity of Ni for the hydrogen evolution reaction (HER). Herein, Ni nanograins incorporated with trace Rh (denoted as 3D RhNi) are synthesized through combining a facile ion-absorption and subsequent thermal reduction treatment, in which Rh increases the local charge density of Ni sites, dramatically accelerating the electrochemical kinetics of HER. Experimentally, the Rh–Ni catalyst with only 0.1 at. % Rh element exhibits superior HER activity among the reported Ni-based materials, whose overpotential is only 37 mV and 18 mV at 10 mA/cm2 in 1.0 M KOH and 0.5 M H2SO4, respectively. DFT calculations further identify that the resultant catalyst possesses appropriate intermediate binding energy in both acidic and alkaline conditions toward HER. This study provides a strategy to manipulate the local electronic structure for significantly improving the activity in surface catalysis.