Introduction: People receiving peritoneal dialysis experience physical function decline, impairing their ability to complete everyday activities, leading to poorer quality of life. Physical factors, including cardiorespiratory fitness, muscle strength, physical activity, and sedentary behavior are associated with physical function. However, little is known about this relationship, or temporal changes of these factors in this cohort. This study aimed to explore modifiable physical factors that are associated with physical function, identify which factor has the strongest influence, and explore temporal changes. Methods: Adults receiving peritoneal dialysis underwent objective and self-reported physical function, cardiorespiratory fitness, muscle strength, physical activity and sedentary behavior assessments 3 times over a 12-month observation period (at baseline, 6 months, and 12 months). Results: Eighty-two participants underwent assessments. All modifiable physical factors were predominantly moderate to strongly associated with physical function at baseline. Cardiorespiratory fitness had the strongest and most consistent influence with every meter conferring a 0.08-unit (P < 0.01) and 0.01-unit (P < 0.05) increase in self-report and objective physical function score, respectively. Temporal changes were observed for modifiable physical factors with significant mean changes in cardiorespiratory fitness (−9.8%), quadricep strength (−5%), moderate-to-vigorous (−25.9%) and total (−16.2%) physical activity, and sedentary behavior (+7.1%). Conclusion: The results of this study indicate that cardiorespiratory fitness could be routinely monitored to detect risk of physical function decline and targeted through intervention to enhance physical function for people receiving peritoneal dialysis. Nevertheless, all factors should be considered when designing interventions to mitigate temporal changes and induce the numerous health benefits offered by being physically active.