Objectives: We developed a patient-level metric to quantify the frequency of documentation of AE assessment in oncology EHR-derived datasets, termed AE assessment completeness score (AACS), and evaluated its utility in a cohort of patients with metastatic breast cancer (mBC). Methods: We included 822 patients with mBC from the Flatiron Health EHR-derived de-identified database receiving palbociclib with endocrine therapy on or before 12/31/2018. We defined expected assessment windows (every 2 weeks for the first 2 months after starting palbociclib; every 4 weeks thereafter), based on palbociclib product information. AACS was calculated as the proportion of expected assessment windows with a documented assessment and dichotomized AACS as high (>75%) or low (≤75%). We calculated the prevalence of AEs of interest (via laboratory data: grades 1-4 elevated AST/ALT and grades 3/4 neutropenia [CTCAE v5.0]; via clinical documentation: first occurrence of diarrhea and deep vein thrombosis [DVT]), stratified by high/low AACS, and compared results to PALOMA-2. Results: 548 patients (67%) had ≥1 selected AE. More patients had high AACS for visits (80%; IQR: 80-100%) than for laboratory tests (ALT/AST/neutropenia: 28%/29%/39%; IQR: ∼40-80%). Patients with high AACS for any AE assessment were older and had longer palbociclib usage (both p<0.01 vs low AACS). AE prevalences were higher in patients with high vs low AACS and were more comparable to those from PALOMA-2 (ALT: 32% vs 18% [43% in PALOMA-2]; AST: 36% vs 24% [52%]; neutropenia: 56% vs 27% [68%]; Diarrhea: 19% vs 16% [26%]; DVT: 3% vs 2% [N/A]). Differences were more pronounced in laboratory-based as opposed to visit-based clinical-documentation of AEs. Conclusions: Incorporation of AACS into studies of AEs using EHR-derived data could facilitate interpretability. In particular, assessment frequency may have a notable impact on prevalence estimates for laboratory-based documented AEs.