Competencies
Patient Care - Gather data; order diagnostic tests; interpret data; make decisions; perform procedures; manage patient therapies; work with others to provide patient-focused care
Medical Knowledge - Fund of knowledge; active use of knowledge to solve medical problems
Practice-Based Learning & Improvement - Analyze practice performance and carry out needed improvements; locate and apply scientific evidence to the care of patients; critically appraise the scientific literature; use the computer to support learning and patient care; facilitate the learning of other health care professionals
Interpersonal & Communication Skills - Develop a therapeutic relationship with patients and their families; use verbal and non-verbal skills to communicate effectively with patients and their families; work effectively as a team member or leader
Professionalism - Demonstrate integrity and honesty; accept responsibility; act in the best interest of the patient; demonstrate sensitivity to patients' ethnicity, age, and disabilities
Systems-Based Practice - Demonstrate awareness of interdependencies in the health care system that affect quality of care; provide cost-effective care; advocate for quality patient care; work with hospital management and interdisciplinary teams to improve patient care
Assessment Methods
Anatomic or animal models - Form of low-tech simulation; performance of procedures using non-computerized 3-dimensional anatomic models that replicate properties of human anatomical structures is observed and assessed concurrently (direct observation). May be used to identify learning gaps and to provide evidence of improvement.
Direct observation - Assessor watches a clinical encounter and assesses performance (same day) using a rating tool specific for the type of encounter (e.g., procedure, physical exam, patient interview, etc.). Direct observation is used as part of other assessment methods, e.g., simulations and OSCE’s. Examples of rating tools are communication checklists (e.g. SEGUE), technical/procedure skills forms (e.g. OSATS, DOPS), clinical encounter forms (e.g., mini CEX).
Formal oral examination - Examiner asks a list of questions in a prepared format about what to do in a clinical scenario presented verbally or role played by the examiner; criteria for evaluation are preset in right/wrong format. High stakes oral exams may be videotaped for documentation purposes.
Global assessment - Rating of overall functioning of resident during a specified time period on a continuum from low to high (specific rating anchors vary) across one or more performance domains; performance descriptors may vary from a single word to multiple sentences. Alternate name for this method is “clinical performance rating.” Global assessments are not specific enough to be used to provide specific feedback to improve resident performance.
In-house written examination - Exam (usually multiple-choice) developed by local (program) faculty; may be administered in either paper or computer format. Since validity and reliability is usually low, these should not be used for high stakes decisions (pass-fail or promotion).
In-training examination - Exam (usually multiple choice) developed by an external vendor; may be administered in either paper or computer format. Validity and reliability will vary depending on the vendor; most ITE’s are intended to be used only for self assessment, program evaluation, and to identify remediation needs.
Multisource assessment - Rating of performance/development from multiple points of view, including two or more of the following: supervisor, peer, subordinate, patient, self. MSA is a limited form of 360-degree assessment. A 360-degree assessment must include all relevant raters, including self-assessment.
Objective structured clinical examination - Consists of several short (5-10 minute) stations; each station is a carefully designed and replicated (standardized) clinical situation (may involve a simulated patient, anatomical models, data interpretation, or other clinical task) and standardized rating tools (standard questions and marking scheme) intended to enable a fairer comparison of peers and assessment of complex procedures without endangering a real patient’s health or safety.
Oral Examination - May involve one or more examiners simultaneously; questions are less structured and evaluations are usually more subjective. Criteria for answers are often less exact and value is often added for problem solving analysis and method, as well as interpersonal communication and presentation. Since validity and reliability is usually low, informal oral exams should not be used for high-stakes decisions (pass-fail or promotion).
Patient survey - Intended to record the patient perspective of a clinical encounter (could be one part of a multisource assessment). If used to assess resident performance, patient surveys must be carefully designed and administered so that aspects of the clinical encounter not under the resident’s control (e.g., parking, registration, billing, etc.) do not affect the rating scores and so that the patient is rating the correct care provider.
Practice/billing audit - Review of billing data of an individual resident which is then compared with practice guidelines and/or peers in the office, hospital or managed care setting. Recommended practice is for one person to be responsible for all audits, review 5 or more records per payor using multiple methods for drawing a random sample, and use a claim analysis checklist to identify appropriateness of coding, documentation and completeness of claim. See Record/chart review (other).
Project assessment - A project is assessed using a standardized rating tool that reflects the learning objectives established for the project. Examples of projects include targeted literature reviews (EBM), root cause analysis, quality improvement, and research.
Record/chart view - Review of patient records to collect data on specified aspects of patient care; used to assess quality of care against pre-specified criteria (e.g., expectations for pain management, physical exam, patient history, use of laboratory tests, etc.). Record reviews may focus on a resident’s performance over time, over a range of circumstances, and/or types of patients. Record reviews are best used for formative assessment. (Also known as medical record audit; clinical care audit.)
Resident experience narrative - Resident records their perspectives of a patient care experience (often a critical incident) and discusses with a faculty mentor. Method should be used only for formative feedback.
Review of case or procedure log - Residents record cases or procedures following a preset protocol. Selected logs are assessed using standardized rating tools that incorporate pre-specified criteria. Batches of logs may be reviewed to determine if program expectations for number and type have been met. Batches of completed assessments may also be reviewed and subjected to global assessment. (See Global assessment.)
Review of drug prescribing - Review of drugs prescribed by an individual resident which is then compared with practice guidelines and/or peers in the office, hospital or managed care setting. Reviews may target specific aspects of drug prescribing such as unnecessary prescribing, cost-effectiveness of prescribing, adequate documentation of information related to prescribing, patient education and counseling related to drug prescribing, etc. See Record/chart review (other).
Review of patient outcomes - Review of patient outcome data which is then evaluated using a set of pre-specified criteria for measurable outcomes (e.g., readmission, relapse, postoperative pneumonia, catheter-associated UTI, etc.) which is then compared to practice guidelines and/or peers in the office, hospital or managed care setting. See Record/chart review (other).
Role-play or simulations - Form of low-tech simulation usually carried out as part of a focused didactic teaching session; assessment is usually narrative, involving verbal self-assessment and feedback from observers of the role-play; used primarily to identify learning gaps.
Simulations/models - Resident performance of procedures on a high-tech computerized simulator is evaluated; may also be used to evaluate individuals within a team and overall team performance during a predetermined clinical scenario. Evaluation data may be collected by the simulator (preprogrammed) or by trained observers via direct observation using standardized tools.
Standardized patient examination - Form of low-tech simulation involving a simulated patient (trained actor) and standardized rating tools (standard questions and marking scheme) for formative assessment of focused or complete patient examination skills. May be used to identify learning gaps and to provide evidence of improvement.
Structured case discussions - Form of informal mini-oral exam; may consist of a small set of pre-determined questions and is commonly conducted as part of a resident’s case presentation to the faculty supervisor or mentor. See Oral examination (informal).
Video Recorded assessment - Resident performance of a clinical procedure or patient encounter is video recorded for later review and concurrent evaluation by a faculty supervisor or mentor; may also include self assessment by the resident and/or assessment by other clinical personnel involved in the procedure or patient encounter (e.g., nurses, peers, other clinicians) after review of the videotape/recording.
Educational Activities
Clinical Teaching - teaching that occurs in the clinic, EDs, ORs, laboratories, or other medical settings and addresses issues related to residents’ current patient cases or clinical responsibilities.
Focused or Direct Observation and Evaluation - direct, hands-on clinical or patient care activities. This may include surgery, patient exams, the reading of radiographs and preparation of pathology assays.
Performance Feedback - information provided to a resident that describes what (s)he has done well or poorly and provides specific guidance as to how performance might be improved.
Departmental Conferences, Lectures or Discussions - formal, classroom instruction on a specific topic or method, led by one or more faculty, residents, or staff, etc.
Institutional Conferences, Lectures, or Discussions - formal educational event involving institution-sponsored grand rounds, lectures, discussions, or workshops for residents and/or faculty from multiple specialties; may be part of an institutional core curriculum (i.e. a set or course of learning activities arranged to impart knowledge and skills in fundamental domains, for example, communication skills, legal issues, ethics).
Individual or Group Projects - multi-step, multi-component tasks performed as vehicles for learning and applying knowledge and skills. Projects should result in a product. Examples are literature reviews, research, clinical quality improvement projects, and community health advocacy work.
Computer Modules - computer-based instructional units that present medical knowledge or clinical tasks, etc, that residents work through independently. These modules are developed either by the institution/program or purchased from commercial vendors.
Standardized Patients - professional actors or real patients trained to present realistically and reliably a medical condition and/or specific patient behaviors; the standardized patient provides instruction to the resident or feedback about his/her performance
High-Tech Simulators/Simulations - 3-dimensional, high tech, computerized devices that represent human anatomy and physiological responses (simulators) are used by residents to learn procedures and operations. Or realistic patient care scenarios are generated using high tech/virtual reality devices (simulations). Residents engage in the scenario as in real life to learn or apply clinical or teamwork skills.
Anatomic or Animal Models - non-computerized, 3-dimensional devices that replicate the properties of human anatomical structures are used by residents to learn procedures.
Role Play or Simulations - staged replicas of potentially real situations are engaged in by residents to learn, practice or rehearse skills needed in those situations. This method is often used in difficult or high-risk situations, e.g. mobilization of a medical team in a multi-victim accident or confrontation of an “impaired “ colleague.
Games - informal activities with goals, rules, rewards and penalties for various courses of action. Games may be computerized, played individually or in groups, facilitated or self-paced.
Role Modeling - portrayal of desired professional behaviors, communication skills, or clinical skills, etc. by attending/supervising physician with the expectation that residents will learn these behaviors and skills by observing the role models.