Taking high-quality photos of digital screens is difficult, as such photos are usually contaminated with moire patterns. Considering the nature of wide-range frequencies of moire' patterns, existing works adopt the multi-scale framework to address this challenge. However, the relationship among feature maps at different scales is significantly ignored, resulting in the degraded performance due to the missing of the semantic information. In this paper, we propose a novel Multi-Scale Feature Enhancing network, named MSFE. By virtue of the multi-scale architecture for extracting moire-irrelevant contexts from multiple resolutions. Furthermore, we design a Feature Enhancing Branch (FEB) to combine high-level features with low-level ones for modeling the correlations of multiple scales. In this way, features with richer semantic information can be learned at each scale. Consequently, moire' patterns at different levels can be tackled properly. Experiments on the publicly moire pattern dataset demonstrate that the proposed method outperforms the state-of-the-arts.