Sediment nitrification plays a vital role in nitrogen (N) biogeochemical cycling and ecological function of an aquatic ecosystem. The relative importance of environmental factors and nitrifying microbial communities in regulating sediment nitrification process has received less attention, especially in aquatic habitats where high N loads are frequently detected. Here, we report the potential nitrification rates of 35 sediment samples from 10 shallow lakes in the Yangtze River basin. The diversity and abundance of ammonia-oxidizing archaea (AOA) and bacteria (AOB) were quantified using archaeal and bacterial amoA genes. The results showed that there was no significant difference in sediment nitrification rates among sites of different trophic state. The nitrification rates were positively related to water chlorophyll-a, sediment N and carbon levels, but not significantly associated with diversity and abundance of ammonia-oxidizing microorganisms and submerged plants. Interestingly, the abundance and diversity of sediment AOB but not AOA communities were significantly influenced by trophic state. In addition, AOB communities were more sensitive to changes in local environments and catchment land uses than the AOA communities. Using path analysis, we found that 55–60% of the indirect effect of catchment land uses on nitrification rates was mediated via sediment N content. Our findings suggest that, although nitrification is a microbial process, variation in sediment nitrification rates in Yangtze lakes is mainly explained by abiotic factors but not by microbial abundance and diversity.