Purpose Internationally, there is increasing interest in monitoring and evaluating doctors' professional practice. Multisource feedback (MSF) offers one way of collecting information about doctors' performance. The authors investigated the psychometric properties of two questionnaires developed for this purpose and explored the biases that may exist within data collected via such instruments. Method A cross-sectional study was conducted in 11 UK health care organizations during 2008-2011. Patients (n = 30,333) and colleagues (n = 17,012) rated the professional performance of 1,065 practicing doctors, using the General Medical Council Patient Questionnaire (PQ) and Colleague Questionnaire (CQ). The psychometric properties of the questionnaires were assessed, and regression modeling was used to explore factors that influenced patient and colleague responses on the core questionnaire items. Results Although the questionnaires demonstrated satisfactory internal consistency, test-retest reliability, and convergent validity, patient and colleague ratings were highly skewed toward favorable impressions of doctor performance. At least 34 PQs and 15 CQs were required to achieve acceptable reliability (G > 0.70). Item ratings were influenced by characteristics of the patient and colleague respondents and the context in which their feedback was provided. Conclusions The PQ and CQ are acceptable for the provision of formative feedback on a doctor's professional practice within an appraisal process. However, biases identified in the questionnaire data suggest that caution is required when interpreting and acting on this type of information. MSF derived from these questionnaires should not be used in isolation to inform decisions about a doctor's fitness to practice medicine.