Friday, January 29, 2021

Educationally Sensitive Patient Outcomes (ESPOs): The Holy Grail for Transformation of Medical Education Research

From the 1/29/2021 newsletter



Director’s Corner

 

 

Educationally Sensitive Patient Outcomes (ESPOs): The Holy Grail for Transformation of Medical Education Research 

 

 

Adina Kalet, MD MPH

 

 

In this Director’s Corner, Dr. Kalet introduces our community to ESPOs and reports on the Kern Institute’s first Invitational International Conference on Medical Education and Patient Outcomes …

 

 

 


Medical Education Research is a young science. When I started out, this work was mostly done by front line (and very busy) medical educators working with a few learning-scientist colleagues. We studied how the structure and process of learning – and individual characteristics of learners and instructional designs – affected individual student learning outcomes. Most studies were small, single institution, and predominately cross-sectional projects that had no comparisons or controls; not surprisingly, our literature was roundly criticized for a lack of rigor and for focusing on questions that had “marginal significance for actual practice(Ouch!). In retrospect, these criticisms were correct and, by the way, who wants to do “marginally significant” work? We knew we could do better. 

 

Many experts in the field vehemently pushed back, though, insisting that doing better was not possible, that there wasn’t enough funding, that studying a physician’s impact on health was too complex, and that rigorous studies would take the “art” out of the practice. I disagreed with these experts and, as you know, I love a good challenge.

 

About ten years ago, my colleagues and I wrote a paper calling on medical education researchers to step up our game. We argued that it was possible to identify learnable, teachable, and measurable “intermediate outcomes” of medical education that were directly linked to good outcomes for patients and populations. We called these Educationally Sensitive Patient Outcomes (ESPOs) and proposed three measurable parameters: Patient Activation, Microsystem Activation, and Health Literacy (Kalet, 2010Yin 2015). We then set to work to systematically test the theory. It was slow going. 

 

When you publish a paper, you are joining a conversation. You take turns, building on each other’s work, critiquing and debating. If you are lucky this leads to “cold calls,” a cup of coffee at professional meeting and, before you know it, you have friends all over the world who share your world view!  

 

After a decade of ESPOs conversations, the Kern Institute sponsored the first, of hopefully many, Invitational Medical Education and Patient Outcomes Conferences this past Monday. Zoom enabled fifteen of us, representing three distinct research teams, three countries (US, Canada and the Netherlands) and nine institutions including the Medical College of Wisconsin, to meet for three hours to share our work, cross-fertilize ideas, and seek opportunities to build a “collaboratory.”

 

 

Here are the stories these folks shared. 

 

 

Resident Sensitive Quality Measures (RSQMs)

 

Can we measure how well resident physicians provide care to children in the Pediatric Emergency room using data from the electronic medical record (EHR)?  Daniel Schumacher, Pediatric Emergency Medicine Doctor at Cincinnati Children’s Hospital, and his team thought it was worth trying. After working with residents and supervising attendings to identify what data in the EHR actually reflect the resident’s contribution to patient care in three specific clinical situations (asthma, bronchiolitis, and closed head injury), he conducted a series of elegant studies to define these RSQMs. MCW’s own Pediatric Emergency Medicine physician, Abigail Schuh and I, are on the team funded by the National Board of Medical Examiners to “validate” his RSQM model using actual clinical data from MCW and NYU (Smirnova, 2019). The team’s leading data analyst, Saad Chahine, PhD, Associate Professor of Measurement and Assessment at the Faculty of Education, Queen’s University, Ontario, presented early findings. This RSQM model is looking promising. It is likely to generate compelling and motivating real patient outcome data that can be fed back to our residents and program directors, thus ensuring these pediatricians will be ready for independent practice. 

 

 

Measuring Resident Operative Performance 

 

Not surprisingly, some near-graduate surgery residents are not yet ready for unsupervised practice. Surgeon Brian C. George, MD at the University of Michigan and his team, including data scientist Andrew Krumm, PhD, Assistant Professor of Learning Health Sciences, have been studying the value of a just-in-time operative performance assessment collected frictionlessly using a smart phone-based software application (SIMPL). Brian runs the Center for Surgical Training and Research and serves as the Executive Director of the Society for Improving Medical Professional Learning (SIMPL), an international collaborative of 129 surgery training programs (Williams, 2017). Through this network, he has collected huge numbers of directly observed measures of resident performance during procedures that surgeons agree are important –appendectomy, inguinal hernia, cholecystectomy, colectomy among others – and can link these measures to Medicare insurance claims data, assessing for complications and outcomes. With their evolving data analytic sophistication, predictions can be made that help us better educate proceduralists to master their craft.  

 

 

Databased for Research in Education in Academic Medicine (DREAM)

 

In 1948, researchers enrolled 5209 adults from Framingham MA in a heart disease risk study and have been following them and their offspring ever since. Almost everything we know about preventing heart attacks and strokes has emerged from this Framingham Heart Study. As Internists, Dr. Sandra Zabar, the 2020 AAMC Abraham Flexner award winner (I am boasting for my friend) and I were inspired by the study team’s “stick-to-it-iv-ness”!  Starting in 2004, The Program for Medical Education Innovations and Research (PrMEIR) at NYU School of Medicine started our own “Framingham-like study” of medical education. Every year, our team seeks medical students’ permission to collect all their admissions, assessment, and survey data from entry to medical school through residency training and into practice. Over the years, most agree (~85%), and almost 3,000 students and residents have been enrolled. So far, this Databased for Research in Education in Academic Medicine (DREAM) has enabled over seventy-five studies (Gillespie, 2016). Colleen Gillespie, PhD, Associate Professor and Director of Education Quality for the Institute for Innovation in Medical Education at NYU, talked about how this longitudinal data can be  used to build individual and aggregate “learning curves,” showing how, for instance,  clinical communication skills develop over the course of medical school and residency, are predicated by admissions data, influenced by curriculum, and related to the outcomes among the patients these physicians care for early in practice. I’m not sure we will be doing this for the next fifty-six years ourselves but, with some luck, others will. 

 

 

Why this is important

 

These types of studies carry implications far beyond the walls of academic medicine. As we work to transform medical education and help nurture the development of character-driven practicing physicians, we must study the impact of educational innovations and make certain we are studying outcomes that matter. 

 

This work carries societal implications. Currently, US taxpayers invest well over $15 Billion each year in resident education via Direct Graduate Medical Education (DGME) and Indirect Medical Education (IME) support, primarily through Medicare and the Department of Veterans Affairs. We need to understand how best to maximize the country’s return on this investment. 

 

This work carries personal implications, as well. Students take on enormous personal debt and invest the prime years of their lives to pursue their careers. Are our educational interventions effective in helping them take control of their education and making certain they are ready, healthy, and able to safely enter practice?

 

Next Steps

 

As you can imagine, the conversation was lively and could have gone on for a long time if not for Zoom fatigue. The data scientists (including Kern’s own Tavinder Ark, PhD) communed around the potential to aggregate various approaches to data analysis and how recursive systems could feed data back to learners and educational leaders, enabling real, virtuous cycles of learning. The medical educators talked about the value of knowing what really matters to patients, both in terms of their health outcomes and their experiences of care. And, of course, we considered the unintentional downside of using clinical and insurance claims data to measure the “quality” of our novice physicians. While the risks are real, the benefits to patients and learners far outweigh the risks if done with a growth mindset, with careful deliberation, and within ethical guardrails. 

 

We resolved to meet as a group at least one more time and then consider conducting a symposium opened up to the larger community. The Kern Institute will continue to convene small and large groups of medical educators and scholars to “lift” the work and identify promising avenues for transformation. Stay tuned! 

 

 

 

For further reading: 

 

Kalet, AL, Gillespie, CC, Schwartz, MD, Holmboe, ES, Ark, TK, Jay, M, ... & Gourevitch, MN (2010). New measures to establish the evidence base for medical education: Identifying educationally sensitive patient outcomes. Academic Medicine, 85(5), 844-851. 

 

Yin, HS, Jay, M, Maness, L, Zabar, S, & Kalet, A (2015). Health literacy: An educationally sensitive patient outcome. Journal of General Internal Medicine, 30(9), 1363-1368.

 

Smirnova, A, Sebok-Syer, SS, Chahine, S, Kalet, AL, Tamblyn, R, Lombarts, KM, ... & Schumacher, DJ (2019). Defining and adopting clinical performance measures in graduate medical education: Where are we now and where are we going? Academic Medicine, 94(5), 671-677.

 

Williams, RG, George, BC, Meyerson, SL, Bohnen, JD, Dunnington, GL, Schuller, MC, ... & Collaborative, S (2017). What factors influence attending surgeon decisions about resident autonomy in the operating room? Surgery, 162(6), 1314-1319.

 

Gillespie, C, Zabar, S, Altshuler, L, Fox, J, Pusic, M, Xu, J, & Kalet, A (2016). The Research on Medical Education Outcomes (ROMEO) Registry: Addressing ethical and practical challenges of using “bigger,” longitudinal educational data. Academic Medicine, 91(5), 690-695.

 

 

 

 

Adina Kalet, MD MPH is the Director of the Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education and holder of the Stephen and Shelagh Roell Endowed Chair at the Medical College of Wisconsin.

Wednesday, January 27, 2021

Dartmouth's Arts & Humanities in Healthcare - 1/29/2021 8:00 - 2:00 CST

 Dartmouth's Arts & Humanities in Healthcare - 1/29/2021 8:00 - 2:00 CST


Here is a link to a day celebrating Dartmouth's Medical Humanities work and research.



Dartmouth-Hitchcock 2nd Annual Arts and Humanities in Healthcare Symposium

Symposium link will be sent out the morning of the event.

Schedule


NOTE: TIMES LISTED ARE LOCAL TIMES IN VERMONT

9:45  Musical prelude Margaret Stephens, Therapeutic Harpist, Dartmouth-Hitchcock

10:00  Welcome and introductions
Lara Ronan, MD, FAAN Associate Professor of Neurology and Medicine Geisel School of Medicine, Vice Chair of Education Dartmouth-Hitchcock Department of Neurology

10:05  Creative arts intervention for patients with refractory epilepsy: A preliminary report
Lara Ronan, MD, FAAN Associate Professor of Neurology and Medicine Geisel School of Medicine, Vice Chair of Education Dartmouth-Hitchcock Department of Neurology

10:15  Telling Our Stories Reinvented: Turning an in-person event into an engaging virtual experience
Andrea Buccellato, Manager, Patient and Family Support Services, Dartmouth-Hitchcock Norris Cotton Cancer Center

10:25  ‘Art at Work’ Repurposed: Adaptations and accomplishments
Marion L. Cate, MEd, CHES, CWWPM, CHC Manager, Health Improvement Program, Employee Wellness, Instructor in Medicine Geisel School of Medicine

10:35  Virtual Perspectives: Art and conversation for people living with memory loss
Claire Lyon, Docent Hood Museum of Art Neely McNulty,
Hood Foundation Associate Curator of Education

10:50  Expressive writing exercise
Marv Klassen-Landis, Creative Writing Specialist, Dartmouth-Hitchcock

11:05  Picturing Contagion: Contextualizing visual iconographies around COVID-19
Emily Luy Tan, Dartmouth College, Class of 2020

11:25  Evidence Based Design in Health: A COVID College Semester 
Erin McGee Ferrell, Professional Artist, Art Educator, University of New England Cancer Patient Advocate National Cancer Institute

11:40  Science, Values and the Novel
Alan Hartford, MD, PhD, Associate Professor of Medicine, Geisel School of Medicine

11:55  Lunch break and musical prelude
Margaret Stephens, Therapeutic Harpist, Dartmouth-Hitchcock

12:15  Keynote: Arts in health research in a post-COVID world
Jill Sonke, PhD, Director, University of Florida Center for Arts in Medicine

1:15  Passages of Writes: Medical students fostering connections through shared reading
Christopher LaRocca, MD, FAAFP, Clinical Associate Professor of Community and Family Medicine, Geisel School of Medicine, Aya Bashi, MS2,Lindsay Becker, MS2, Rachel Brown, MS2, Zachary Panton, MS2

1:30  Mindful art project
Kim Wenger Hall, Visual Artist, Dartmouth-Hitchcock

1:45  What about clinician burnout, anxiety, and PTSD during COVID-19? What are the arts bringing to support them?
Alan Siegel, MD, Director, Art of Health and Healing, Founding Board Member NOAH

2:15  2020 Arts-based initiatives at Walter Reed National Military Medical Center
Mallory Van Fossen, ATR-BC, LCPAT, LPC, Art Therapist, Clinical Coordinator of Walter Reed National Military Medical Center’s Arts in Health Program

2:45  Closing remarks

Learning Outcome:

At the conclusion of this learning activity, participants will be able to recognize the value and impact of the arts and humanities on health and well being.

Monday, January 25, 2021

How “Mattering” Matters

 From the 1/22/2021 newsletter


Perspective

 

 

How “Mattering” Matters

 

 

Karen Marcdante, MD

 

 

Dr. Marcdante writes about the important and emerging concept of “mattering” in medical education and patient care …




It had been quite a week in the Pediatric ICU, lots of patients with complex issues and a few sad stories.  One such story involved a young girl who had been injured in a car accident.  We had just confirmed that she was now brain dead.  I sat down with her mother to explain the findings.  We spent some time talking about her daughter, who she was as a person, her joys and some of her challenges.  As we walked down the hall, back to her daughter’s room, the mother reached out to hold my hand.  We entered the room and shared a few tears as we knew what was to happen soon.  

 

This day, while encompassing the hardest part of my job, stays with me.  Why?  I left that day, knowing that I had mattered, knowing that, despite this being one of the worst days of that family’s life, I had made a difference.  

 

My forty-year career as a physician is, fortunately, full of such moments, little things that remind us of the value we bring and reaffirming our perception that we have something to contribute.   Those moments play a role in my love of my job and keeping burnout at bay.  There are also moments when I felt I didn’t matter – like when I am ignored or made to feel out of place, when my voice isn’t heard or when someone tries to micromanage (what is the point, if they are just going to redo whatever I did?).  As I look to yet another transition in my career, I have looked closely at the idea of mattering.

 

 

Mattering, in fact, is a psychological concept that involves relationships.  It is the perception that others are aware of my presence and strengths, that they rely on me and that they see me as important to them, the team, or even to society.  While it is something I perceive, it really is about relationships – as we all have a need to feel that we matter to others.  It is, in many ways, transactional as mattering is about an exchange between people where “the others’” words, actions and behaviors lead me to perceive that I matter (or don’t).  It is also reciprocal, as my words, actions and behaviors help others perceive they matter.  First described by Rosenberg and McCullough in 1981, more recent studies have demonstrated that when people feel they matter they experience a greater sense of well-being and belonging, more self-esteem and self-efficacy, and even learn better.  People who feel that they matter experience less burnout and depression.  Organizations where people feel they matter are more productive with less personnel turnover.  

 

 

Three components of Mattering

 

So, as I think about how I can still be of value and contribute, I also have been realizing just how important making people feel that they matter can be.  And it doesn’t take a lot of resources, especially to address the interpersonal mattering.  Let’s look at the three major components of mattering to identify what we can do.  

 

 

Awareness is the first component.  Recent interviews with medical students reveal that one way to make them matter is to learn (and call them by) their names.  It can be that simple.  Asking a few questions to learn more about each other is another way – one you likely already do but that you may skip if you are busy or if the other person is not likely to spend much time with you.  Saying hi in the hallway or asking, kindly, about an absence are other ways to make people feel they matter. 

 

Importance, as a component of mattering, means that someone expresses concern for you.  Again, simple things may help.  One of the simplest – but not always easiest – is investing time in people.  I recently spoke with one of our colleagues who focuses on his role as advisor by talking with students for several hours over time (not just in a single session, reaching out as they prepare for and go to their interview.  The students must feel that they matter (and, hopefully,  so does the faculty member when he sees the students’ successes!).  Advocating for others is another way to make them feel that they matter.  So is providing appropriate emotional support – often just listening to them during challenging times.   Others know they are important when you provide growth-directed feedback, especially when they need to improve performance.    

 

The third component of mattering is reliance – how others look to you for your help and skills (your contributions).  You know you matter when others rely on you to “be there,” complete tasks, or seek your advice.   A simple way to make people realize that you rely on them is to seek their input on decisions that impact them.  Acknowledging that you missed someone (not just because work didn’t get done) is perceived as a form of reliance.    Granting learners autonomy, especially if it pushes them a little out of their comfort zone, is one way to let them know you are relying on them.  

 

 

Building relationships

 

As you can see, much of interpersonal mattering is about building a trusting and reciprocal relationship.  You also want to matter to your organization.  For this, organizations need to have effective ways to demonstrate awareness (recognition, whether a supervisor’s thank you or an institutional award for a sustained contribution); importance (seeking your input, positively addressing inclusion, diversity and equity, or listening and responding to employee concerns); and reliance (making expectations clear with appropriate accountability, allowing you to use your strengths).   Organizational mattering is about building systems that support and enhance the relationship between you and the organization.  It may be harder in some ways and require more resources but being explicitly aware of how people feel they matter and incorporating ideas that help them perceive mattering more regularly can improve the work environment.

 

What I have learned as I have studied mattering is that it seems to be one of the foundational concepts, one we may take for granted or not pay attention to as we deal with all the changes and stresses of both the typical and the post-COVID-19 work environment.  I have committed to being more explicit in making people feel they matter and hope that we can find some systemic solutions that allow us all to do the same.  I wonder what would happen if we thought each day about how to make those we will interact with feel they matter – whether it is your patients, your coworkers, or your family.  Research suggests that the work environment will feel safer, and that we will be more creative, happier, and more productive.  I say, let’s try!

 

 

 

Karen Marcdante, MD is a Professor in the Department of Pediatrics (Critical Care) at MCW. She is Director of the Human Centered Design Laboratory,  a member of the Faculty Pillar and serves on the steering committees of the KINETIC3Teaching Academy and Philosophy of Medical Education Transformation Laboratory, all of the Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education. 

 

 

Three Questions for Dr. Mary Horowitz about Mentoring

From the 1/22/2021 newsletter


‘Take 3’ 


Three Questions for Dr. Mary Horowitz about Mentoring


Dr. Mary Horowitz, long-time MCW Faculty Member, Professor and Chief Scientific Director of the Center for International Blood &Marrow Transplant Research (CIBMTR) shares candid thoughts about her experience with mentoring.  Dr. Horowitz has clinical and research interests in quality of life for transplant recipients and graft vs host disease, and has led incredible efforts in data sharing across bone marrow transplant centers nationally and internationally.  She has been the recipient of the MCW Distinguished Service Award, the American Society of Hematology’s Mentor Award for Clinical Science and a Lifetime Achievement Award from the American Society of Blood and Marrow Transplantation.  



What has surprised you most, personally, in your many years of dedicated mentoring? 

The joy I felt at my mentees’ accomplishments! I, of course, wanted them to be successful and expected to be pleased – but the degree to which their success made me happy was unexpected.


What are some common mistakes to avoid for our junior faculty who are more novice in their mentoring skills?

Don’t try to solve all of your mentee’s problems for them – and be willing to accept solutions that might not necessarily be the way you would do it. There is usually more than one good way to attack a problem. On the other hand, don’t be afraid to get granular when necessary – for example, if you see a mentee is spending too much time on committees and other service responsibilities and not enough on academic advancement, don’t just say “you are over-committed” – go through their lists of activities with them and have them decide which ones should be dropped.


Can you share a story of a successful mentoring relationship you have had, and what made it work

I have several long-term mentoring relationships that have evolved into friendships – and are now relationships where the mentoring is bidirectional. The key, I think, is to really care.  It has always been really important to me that my mentees be successful – and not only in their professional life. You cannot ignore the signs that personal issues are affecting professional responsibilities – and you shouldn’t be afraid to ask. Doesn’t mean you should pry – some people are very private – but acknowledging that our lives include more than work and that everyone sometimes has to divert their attention to things outside the workplace can relieve some of the stress and actually lead to more productivity. A little organization helps also. I started many years ago asking mentees to make a list of things to discuss during our meetings – it made the meetings much more productive and made sure that we focused on the things that mattered to them. And, as the mentee and the relationship matures, I started asking their advice on things.