For robotics and AI applications, automatic facial expression recognition can be used to measure user’s satisfaction on products and services that are provided through the human-computer interactions. Large-scale datasets are essentially required to construct a robust deep learning model, which leads to increased training computation cost and duration. This requirement is of particular issue when the training is supposed to be performed on an ongoing basis in devices with limited computation capacity, such as humanoid robots. Knowledge transfer has become a commonly used technique to adapt existing models and speed-up training process by supporting refinements on the existing parameters and weights for the target task. However, most state-of-the-art facial expression recognition models are still based on a single stage training (train at once), which would not be enough for achieving a satisfactory performance in real world scenarios. This paper proposes a knowledge transfer method to support learning using cross-domain datasets, from generic to specific domain. The experimental results demonstrate that shorter and incremental training using smaller-gap-domain from cross-domain datasets can achieve a comparable performance to training using a single large dataset from the target domain.