Precise and resilient quantum gate design is important for the building of quantum devices. In this paper, we consider the optimal and robust quantum gate design problem for three classes of two-level quantum systems. The aim is to construct quantum gates in a given fixed time with limited control resources. A modified dueling deep Q-learning (MDuDQL) is employed for the optimal and robust gate design problem. To improve the performance of the classical DuDQL method, we propose a unique semi-Markov DuDQL algorithm based on a modified action selection procedure, modified replay memory, and soft update procedure. The proposed algorithm outperforms ordinary DuDQL in terms of discovering global optimal or near-global optimal control protocols and faster convergence to a better policy. Moreover, the modified DuDQL agent shows improved performance in finding robust control protocols which achieve high-fidelity quantum gate design for varying uncertainties in a certain range. The effectiveness of the proposed algorithm for the optimal and robust gate design problems has been illustrated by numerical results.