Robust optimisation refers to the process of finding optimal solutions that have the lowest sensitivity to possible perturbations. In a multi-objective search space the robust optimal solutions should have the least dispersion on all of the objectives, making it a more challenging problem than in a single-objective search space. This paper establishes a novel and cheap technique for finding robust optimal solutions called confidence-based robust multi-objective optimisation. This approach uses a novel, modified Pareto dominance operator to differentiate search agents of meta-heuristics based on both levels of robustness and confidence. The proposed confidence-based Pareto dominance allows us to design different confidence-based robust optimisation variants of meta-heuristics based on different methods. As a case study, robust Multi-Objective Particle Swarm Optimisation is equipped with the proposed operator to produce Confidence-based Robust Multi-Objective Particle Swarm Optimisation. A set of specific test functions and performance indicators is employed for benchmarking the Confidence-based Robust Multi-Objective Particle Swarm Optimisation. The results show that the proposed method is able to confidently and reliably find robust optimal solutions without significant extra computational burden. The paper also considers finding the robust Pareto optimal front for a marine propeller design problem to demonstrate the applicability of the approach proposed in solving computationally expensive real-world problems with unknown true robust Pareto optimal fronts.