Object detection is an emerging and essential problem in recent years, which has been widely applied in many aspects of daily life such as video surveillance, self-driving robots, and automatic payment. The rapid development of deep learning models allows object detectors to work in real-time with high accuracy. However, such a sophisticated model often requires robust computing infrastructure such as powerful graphics processing units (GPUs). This requirement might cause a severe issue for embedded systems with small, power-efficient artificial intelligence (AI) systems like Jetson Nano, which are often restricted in both memory storage and computing sheer power. In this work, we aim to address this challenge by proposing a lightweight object detection framework that is specialized for the Internet of Things (IoT) devices with low-power processors such as Jetson Nano. In order to detect the object with different size, our framework employs a backbone residual CNN-based network as the feature extractor. We then design a multi-layer model to combine the feature at different levels of granularity, before using the processed feature to locate and classify the object. We also apply augmentation techniques to enhance the robustness of the framework to adversarial factors. Extensive experiments on real devices in many scenarios, such as autonomous cars or wireless robot recharging systems, showed that our technique can achieve nearly on par results with the state-of-the-art YOLOv5 while requires only one-fourth of computation power.