The increasing integration of renewable energy sources into islanded power systems presents significant challenges for frequency regulation, primarily due to their inherent variability and uncertainty. Traditional frequency controllers often rely on fixed parameters or offline tuning methods, limiting their adaptability to dynamic operational conditions. This paper introduces an innovative Optimized Online Adaptive Frequency Control (OAFC) framework, which integrates model-free reinforcement learning with a Gaining-Sharing Knowledge-based Optimization (GSKO) algorithm to address these limitations. The GSKO algorithm systematically determines the initial stabilizing control policy, thereby resolving a key gap in conventional adaptive control schemes. Subsequently, the OAFC updates control gains in real-time through adaptive dynamic programming, eliminating the need for large offline datasets and ensuring robust performance under renewable intermittency and load disturbances. Extensive simulations conducted in MATLAB/Simulink, including sudden load drops and renewable generation fluctuations, confirm that OAFC achieves superior performance, with reductions of 19.16% in root mean square error, 32.41% in mean absolute error, and 34.65% in integral squared error compared to state-of-the-art Online Supplementary Learning Controllers. Furthermore, statistical analysis and robustness testing validate the reliability and practical applicability of the proposed control scheme for future renewable-dominant islanded microgrids.