Author Archives: guestbloggerstfm

Joseph Brocato, PhD

If you are like me, I am sure that many of you are vacillating between denial and avoidance—and perhaps even outright hostility—when contemplating the new ACGME Family Medicine RRC Milestones and Next Accreditation System (see
http://www.acgme-nas.org/family-medicine.html
). While indeed they involve a new way of tracking residents’ attainment of fundamental knowledge, skills, and attitudes, it also suggests that we as faculty need to make efforts to examine how much we know about evaluating our residents in this new era of competency-based education. What are some of the faculty skills we need to hone?
While there are the traditional academic roles of teaching, research, and scholarship and embedded skills within each, we now find ourselves needing to become much more proficient in the area of evaluation: how do we do a thorough evaluation of our trainees, and how do we take a potentially large bolus of evaluation data points for each resident/fellow and make a objective decision about residents/fellows reaching the sometimes seemingly elusive marker of being “competent to practice independently”?

So what are the knowledge, skills, and attitudes concerning evaluation that we need to be competent to administer the NAS and the FM Milestones? In 2006, the International Board of Standards for Training, Performance, and Instruction developed a list of 14 evaluator competencies clustered in four general domains (see:
http://www.ibstpi.org/
)

A.  Professional Foundations
1. Communicate effectively in visual, oral, and written form.
2. Establish and maintain professional credibility.
3. Demonstrate effective interpersonal skills.
4. Observe ethical and legal standards.
5. Demonstrate awareness of the politics of evaluation.

B.  Planning and Designing the Evaluation
6. Develop an effective evaluation plan.
7. Develop a management plan for the evaluation.
8. Devise data collection strategies to support the evaluation questions and design.
9. Pilot test the data collection instruments and procedures.

C. Implementing the Evaluation Plan
10. Collect data.
11. Analyze and interpret data.
12. Disseminate and follow-up the findings and recommendations.

D. Managing the Evaluation
13. Monitor the management plan.
14. Work effectively with personnel and stakeholders.

For the list of 14 evaluator competencies above, the ITSBI has also developed 84 associated performance statements related to each of these competencies. I would posit that the list above would make a good skeletal framework for a family medicine faculty development seminar(s) on evaluation to prepare us for the Milestones and NAS. I can see some real interesting dialogue emerging locally and nationally within residency faculty circles around these evaluator competencies.

My questions for you are these:

(1) Since the ACGME demands a core set of demonstrable residency competencies of our residents and fellows, should we be charged with our own set of evaluator competencies as residency and fellowship faculty?

(2) How ready are we as faculty members to play a larger and more robust role as evaluators?

I am anxious to hear your thoughts on “faculty as evaluators”: where we have been and more importantly, where we need to go!