Deep learning models are emerging to summarize source code to comment for code documentation and program comprehension. We can achieve good performance by training the model on large training corpus. However, in practice, the code samples from different projects can have contradictory training signal for learning a deep comment generator, making the model struggled to fit all the training samples. In this work, we introduce a novel approach, AdaCom, to improve the performance of comment generators by on-the-fly model adaptation. This research is motivated by the observation that deep comment generators often need to strike a balance as they need to fit all the training samples. Specifically, for one certain target code c, some training samples Sp could have made more contributions while other samples So could have counter effects. However, the traditional fine-tuned models need to fit both Sp and So from a global perspective, leading to compromised performance for one certain target code c. In this context, we design AdaCom to (1) detect whether the model might have a compromised performance on a target code c and (2) retrieve a few helpful training samples Sp that have contradictory samples in the training dataset and, (3) adapt the model on the fly by re-training the Sp to strengthen the helpful samples and unlearn the harmful samples. Our extensive experiments on 7 comment generators and 4 public datasets show that (1) AdaCom can significantly boost the performance of comment generation (BLEU4 score by on average 14.9%, METEOR by 12.2%, and ROUGE-L by 7.4%), (2) the adaptation on one code sample is cost-effective and acceptable as an on-the-fly solution, and (3) AdaCom can adapt well on out-of-distribution code samples.