Deep learning model suffers from catastrophic forgetting when learning continuously from stream data. Existing strategies for continual learning suppose the forgetting always happens when learning a new task and only deals with the previous task's global forgetting. This study introduces a novel active forgetting detector based on a windowing technique that monitors the model's forgetting rate for each encountered class label. When the model experiences the forgetting issue, we adapt the forgetting classes by using a proposed replay from experience method called online triplet rehearsal. We conduct comprehensive experiments on four vision datasets to demonstrate that the proposed approach performs significantly better than three state-of-the-art continual learning methods.