Joseph Brocato, PhD
If you are like me, I am sure that many of you are vacillating between denial and avoidance—and perhaps even outright hostility—when contemplating the new ACGME Family Medicine RRC Milestones and Next Accreditation System (see http://www.acgme-nas.org/family-medicine.html). While indeed they involve a new way of tracking residents’ attainment of fundamental knowledge, skills, and attitudes, it also suggests that we as faculty need to make efforts to examine how much we know about evaluating our residents in this new era of competency-based education. What are some of the faculty skills we need to hone?
While there are the traditional academic roles of teaching, research, and scholarship and embedded skills within each, we now find ourselves needing to become much more proficient in the area of evaluation: how do we do a thorough evaluation of our trainees, and how do we take a potentially large bolus of evaluation data points for each resident/fellow and make a objective decision about residents/fellows reaching the sometimes seemingly elusive marker of being “competent to practice independently”?
So what are the knowledge, skills, and attitudes concerning evaluation that we need to be competent to administer the NAS and the FM Milestones? In 2006, the International Board of Standards for Training, Performance, and Instruction developed a list of 14 evaluator competencies clustered in four general domains (see: http://www.ibstpi.org/)
A. Professional Foundations
1. Communicate effectively in visual, oral, and written form.
2. Establish and maintain professional credibility.
3. Demonstrate effective interpersonal skills.
4. Observe ethical and legal standards.
5. Demonstrate awareness of the politics of evaluation.
B. Planning and Designing the Evaluation
6. Develop an effective evaluation plan.
7. Develop a management plan for the evaluation.
8. Devise data collection strategies to support the evaluation questions and design.
9. Pilot test the data collection instruments and procedures.
C. Implementing the Evaluation Plan
10. Collect data.
11. Analyze and interpret data.
12. Disseminate and follow-up the findings and recommendations.
D. Managing the Evaluation
13. Monitor the management plan.
14. Work effectively with personnel and stakeholders.
For the list of 14 evaluator competencies above, the ITSBI has also developed 84 associated performance statements related to each of these competencies. I would posit that the list above would make a good skeletal framework for a family medicine faculty development seminar(s) on evaluation to prepare us for the Milestones and NAS. I can see some real interesting dialogue emerging locally and nationally within residency faculty circles around these evaluator competencies.
My questions for you are these:
(1) Since the ACGME demands a core set of demonstrable residency competencies of our residents and fellows, should we be charged with our own set of evaluator competencies as residency and fellowship faculty?
(2) How ready are we as faculty members to play a larger and more robust role as evaluators?
I am anxious to hear your thoughts on “faculty as evaluators”: where we have been and more importantly, where we need to go!
STFM Blog 
John – VERY interesting questions that you pose in this post about whether family medicine faculty should be charged with demonstrating evaluator competencies? Perhaps we should go one step further and ask who then will be evaluating the evaluators of the residents?
Having been a “resident evaluator” for 30 years when I was fulltime faculty – I totally relate to the issues you pose. Now that I have retired from academia – I have finally had the time to develop a time-efficient approach to objective evaluation and documentation of resident competency in the specific area of my field of special interest = ECG Interpretation. My thoughts to the issues you raise are, “Where does it end?” How can fulltime faculty ever hope to have enough time in the day to not only oversee/supervise residents on all clinical matters – but also evaluate, document, and verify competency in each of the multiple clinical fields – while still performing the numerous other administrative and clinical tasks expected of them as fulltime faculty – not to mention publishing and performance of other educational scholarship?
ECG Competency at least proposes an objective, time-efficient, interesting and educational method for “painlessly” solving this issue of documenting resident competency in the area of ECG interpretation. Perhaps it might serve as a model for others to develop similar tools to assess resident competency in other clinical areas?
DISCLAIMER: I am the developer of ECG Competency, which is a commercial product.
For those who may be interested – Information on ECG Competency is available on my web site: https://www.kg-ekgpress.com/ecg_competency-exams-assessement/
– Information about me and my credentials is available at: https://www.kg-ekgpress.com/about/
– I can be contacted by e-mail: ekgpress@mac.com
– I will be presenting for the 9th consecutive year 2 ECG Interpretation Workshops at the National AAFP Conference for Residents and Medical Students in KC from August 1-3, 2013. I am exhibiting at Booth #1221 at this conference – where I’ll be glad to discuss objective documentation of resident competency in person with you.