Traffic offloading via WiFi networks is an effective way to alleviate drone cellular network congestion. Existing studies are based on a limited model which assumes the presence of only a single drone and a single WiFi network in the region. We study the more general scenario of traffic offloading using multiple WiFi networks in the cellular networks composed of multiple drones which requires collaborative service provision, and address the problem of jointly minimizing the total distance traveled by the drones and the maximum latency. We propose an effective approach to solving this problem by combining optimization and machine learning. We first determine the user-drone allocation by applying meanshift clustering [1], solve the UAV navigation problem by Bellman-Floyd’s minimum-cost maximum-flow algorithm [2], and then deploy reinforcement learning to compute the optimal percentages of traffic offloading for different users. Simulation results demonstrate that our algorithm avoids violating the constraints and keeps the maximum delay in an acceptable range.