The “Grain for Green” (GFG), a large-scale vegetation restoration project, has significantly reduced runoff and sediment yield in the Loess Plateau region of China since 1999. While numerous studies have quantitatively described the relationship between vegetation cover and runoff and sediment yield at the plot scale, there are few such studies, especially those on sediment yield, at the catchment scale. To model the effect of vegetation restoration, soil infiltration capacity and soil erosion were parameterized for the Digital Yellow River Model (DYRIM), which is a distributed watershed runoff and sediment dynamics model that incorporates hillslope erosion, gravitational erosion and sediment transport, and has been widely applied to the Loess Plateau. DYRIM with improved functionality was validated on an event basis and as continuous simulation for the Upper Qingjian River Basin (UQRB, with an area of 913 km2) where the vegetation coverage has increased from 15 % to 70 % during 1986–2015. Simulation agrees well with observation in terms of daily flow (NSE = 0.928 for calibration and NSE = 0.602 for validation) and daily sediment discharge (NSE = 0.938 for calibration and NSE = 0.520 for validation). According to our simulation, the runoff reduced by 44 % and the sediment yield reduced by 82 % for the catchment due to the GFG project from 2003 to 2015. Both runoff and sediment yield decrease exponentially with increasing vegetation cover at large scales, and the effect of vegetation cover diminishes during extreme rainstorm events. When changes in vegetation cover are combined with changes in precipitation, the simulated peak sediment yield occurs with the mean annual precipitation of around 450–500 mm⋅yr−1. The improved DYRIM model can credibly capture and reproduce the effect of the change in vegetation cover on runoff and sediment yield at the catchment and regional scales.