This paper presents a method for rule learning in large knowledge graphs. It consists of an effective sampling of large knowledge graphs (KGs) based on the attention mechanism. The attention-based sampling is designed to reduce the search space of rule extraction and thus to improve efficiency of rule learning for a given target predicate. An implementation ARL (Attention-based Rule Learner) of rule learning for KGs is obtained by combining the new sampling with the advanced rule miner AMIE+. Experiments have been conducted to demonstrate the efficiency and efficacy of our method for both rule learning and KG completion, which show that ARL is very efficient for rule learning in large KGs while the precision is still comparable to major baselines.